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Abstract 

We investigate an approach to physical-layer security based on the premise that the coding mechanism 
for secrecy over noisy channels is fundamentally tied to the notion of resolvabihty. Instead of considering 
capacity-based constructions, which associate to each message a sub-code whose rate approaches the 
capacity of the eavesdropper's channel, we consider resolvability-based constructions, which associate 
to each message a sub-code whose rate is beyond the resolvability of the eavesdropper's channel. We 
provide evidence that resolvability is a more powerful and perhaps more fimdamental coding mechanism 
for secrecy by developing results that hold for strong secrecy metrics and arbitrary channels. Specifically, 
we show that, at least for binary symmetric wiretap channels, random capacity-based constructions fail to 
approach the strong secrecy capacity while resolvabihty-based constructions achieve it. We then obtain the 
secrecy-capacity region of arbitrary broadcast channels with confidential messages and a cost constraint 
for strong secrecy metrics, which generalizes existing results. Finally, we speciahze our results to study 
the secrecy capacity of wireless channels with perfect channel state information, compound and mixed 
channels, as well as the secret-key capacity of source models for secret-key agreement. By tying secrecy 
to resolvability, we obtain achievable rates for stronger secrecy metrics and with simpler proofs than 
previously derived. 

Index Terms 

information-theoretic security, wiretap channel, secret-key agreement, information-spectrum, resolv- 
ability, wireless channels. 
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I. Introduction 

In virtually every communication system, the problems of reliability and secrecy are handled in 
fundamentally different ways. Typically, error-correcting schemes in the physical-layer guarantee reliable 
communications, while encryption algorithms and key-exchange protocols in the upper layers^ ensure data 
secrecy. Physical-layer security puts forward an alternative role for the physical layer, whereby reliabiUty 
and secrecy can be handled jointly by means of appropriate coding schemes. The premise of physical-layer 
security is to recognize the presence of noise in every communication chaimel, including the chaimel of a 
potential adversary who eavesdrops on transmitted signals, and to exploit knowledge of noise statistics to 
prevent eavesdroppers from retrieving information. UnUke usual security schemes, physical-layer security 
can guarantee information-theoretic security, by which secrecy is measured quantitatively in terms of the 
statistical independence between the messages transmitted and the observations of eavesdroppers. 

The theoretical foundations of physical-layer security build upon the early works of Wyner [1] and 
Csiszar & Komer [2], which prove the existence of coding schemes ensuring reliability and secrecy 
for the wiretap chaimel; however, the recent surge of information-theoretic results about the wiretap 
chaimel has fostered few practical engineering solutions. This state of affairs is partly due to the fact 
that most works exploit the coding schemes of [I], [2], in which the coding mechanism that guarantees 
secrecy is tied to chaimel capacity. This mechanism will be precisely defined in Section HI; at this point, 
suffice to say that the codes in [I], [2] are a union of sub-codes whose rates approach the chaimel 
capacity of the eavesdropper's channel as the blocklength grows large. Although such coding schemes 
have been successfully used to study many multiuser information-theoretic secrecy problems [3], [4], 
deriving secrecy from channel capacity leaves open a few lingering issues: 

1) wiretap chaimel models that incorporate the limitations of modern communication systems, such 
as memory or lack of chaimel state knowledge, are difficult to analyze; 

2) the results obtained by tying secrecy to chaimel capacity are deemed too weak for cryptographic 
appUcations. 

This paper discusses an alternative approach to physical-layer security that addresses the aforementioned 
issues; the premise of the approach is that the coding mechanism for secrecy is fundamentally related to 
the notion of resolvability [5] and not to channel capacity. 

'Specific cryptographic schemes are implemented at all upper layers of the protocol stack, including MAC, transport, network, 
and appUcation layers. 
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A. Motivating Examples 

To motivate the approach, we start with two intuitive examples that shed Ught on the mechanisms one 
could exploit to ensure information-theoretic security. 

Example 1 (One-time pad). Consider a binary message M. G {0, 1} that is encoded into a codeword Z 
a* Z = M. K, where K ^ B{p) is a secret key and © denotes the modulo-two addition. If p = \, the 
crypto lemma [6] shows that the output distributions pz|M=o 'S'/fi jOz|M=i ^''^ identical and equal to the 
uniform distribution on {0,1}; hence, messages are statistically indistinguishable for an eavesdropper 
observing Z alone. From an operational perspective, note that the encoder exploits the key K to ensure 
that all messages induce the same output distribution. 

Example 2 (Transmission over a noisy Gaussian channel). Consider an uncoded message M uniformly 
distributed in the set { — 1, +1} and observed by an eavesdropper at the output of a real additive white 
Gaussian noise channel aj Z = M. + N, where N ~ AA(0, o"^). As illustrated in Figure 1, the output 
distributions Pz\M.=-i '^'^d pz\M.=+i become indistinguishable from the average distribution pz as the 
noise variance increases. Specifically, as a goes to infinity, one can show that, for each m.G {— 1;+1}, 
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Fig. 1. Distributions of channel outputs over AWGN channel. 

the variational distance between Pz|M=m and pz satisfies 

/ |PZ|M=m(z) -Pz(2;)|dz = 0(cr"i). 

In other words, if the channel introduces enough randomness, then the channel itself ensures that all 
messages induce approximately the same output distribution. 

In both examples, statistical indistinguishability is obtained because there exists a source of randomness 
(key or channel noise) and a coding mechanism by which all messages induce the same distribution for 

the eavesdropper's observations; this operation is reminiscent of the codes analyzed in [5], [7] to study 
the notion of resolvability. At this point, the connection between secrecy and resolvability may seem 
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contrived but, nevertheless, it suggests the possibihty of ensuring secrecy by means that are radically 
different from those based on channel capacity and used in [1], [2]. In the remainder of this paper, we 
develop a set of results that expand upon the ideas introduced in Example 1 and Example 2. We not only 
highlight the benefits of explicitly connecting secrecy to resolvability but also show the limitations of an 
approach based on channel capacity. 

B. Related Work 

Most communication architectures providing information-theoretic security are based on two models of 
communication. The wiretap channel, introduced by Wyner [1] and generahzed by Csiszar & Komer [2], 
models an architecture in which a transmitter encodes messages M. into codewords X" of n symbols 
for transmission to a receiver, in the presence of an eavesdropper who obtains noisy observations of 
X". In the case of discrete memoryless channels, [1], [2] have shown the existence of coding schemes 
simultaneously ensuring reliable transmission to the receiver and secrecy with respect to the eavesdropper. 
In particular, it is possible to characterize the secrecy capacity of a wiretap channel, defined as the 
supremum of all reliable and secure rates. The extension of this result to Gaussian [8] and wireless 
channels (see, for instance, [9] and references therein) suggests the potential of such coding schemes to 
secure communication networks at the physical layer. An alternative to the wiretap channel is the source 
model for secret-key agreement introduced by Maurer [10] and Ahlswede & Csiszar [11], which considers 
an architecture in which two legitimate parties attempt to distill secret keys from a noisy source by 
communicating over a pubUc channel. The resulting keys have to be secure with respect to an eavesdropper 
who obtains correlated observations from the source and observes all messages exchanged over the public 
channel. This architecture differs from the wiretap channel by exclusively focusing on the rate of secret key 
that can be distilled from the source and by ignoring the cost of public communication. The counterpart 
of secrecy capacity is the secret-key capacity, defined as the supremum rate of secret keys that can 
be distilled. Although the aforementioned architectures model fundamentally different communication 
scenarios, they are related in that a coding scheme for the wiretap channel can be used to design a 
coding scheme for secret-key agreement and vice-versa. 

The information-theoretic security results obtained for the wiretap channel and source model for secret- 
key agreement are criticized in some circles for measuring statistical independence in terms of the rate 
of information leaked to the eavesdropper ^I(M; Z"). The weakness of this metric from a cryptographic 
standpoint has been highlighted in multiple works [4], [12], which have advocated using the total 
amount of information leaked I(M; Z") instead. The analysis of secure communication architectures 
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under this more stringent secrecy metric has been performed with different methods, such as graph- 
coloring techniques [13] and privacy ampUfication [12], [14]. We also note that resolvabihty has already 
been used more or less implicitly in [15], [16]; the results presented in this paper differ from these 
earlier works by making resolvability the explicit mechanism for secrecy and generalizing known results 
to several models, including compound channels and continuous channels with cost constraints. 

The connection between secrecy and resolvability is better highlighted by studying secure commu- 
nication architectures beyond the traditional memoryless setting; in particular, the distinction between 
the coding mechanisms for reliabiUty and secrecy becomes apparent in the expressions of the results 
themselves. In this context, the information- spectrum methods pioneered by Han and Verdu turn out 
to be convenient mathematical tools, as they allow us to analyze general channels by focusing on the 
properties of mutual information as a random variable [5], [7], [17]. We note that these tools have already 
been used to study information-theoretic security beyond memoryless channels and our results provide 
extensions of [15], [18]-[20]. 

C. Summary of Results 

In this section, we highlight the results presented in this paper, preliminary versions of which have 
been reported in [21], [22]. 

• We clarify the relation between information-theoretic security and statistical independence by in- 
vestigating alternatives to the average mutual information rate ^I(M.; Z"), which is used as the de 
facto metric in most earlier works. The average mutual information rate is actually a normahzed 
KuUback-Leibler distance between the joint distribution pmz" and the product distribution pmPz" ; 
the distance between these two distributions can be measured by other means, such as the variational 
distance or even the cumulative distribution function (CDF) of the random variable I(M;Z"). By 
establishing relations among different metrics in Section III, we highlight the importance of choosing 
a measure of statistical independence that is not only simple enough to be analytically tractable but 
also strong enough to be cryptographically relevant. In addition, this discussion provides the basis 
for elegant converse proofs. 

• We provide evidence that resolvability may be the fundamental coding mechanism for secure com- 
munication by making rigorous the ideas suggested in Example 1 and Example 2. Specifically, 
we connect secrecy to resolvability to analyze the fundamental limits of Shannon's cipher system 
(Theorem 1 in Section IV) and of the broadcast channel with confidential messages (Theorem 2 
in Section V). In the later case, we show that, at least for a specific wiretap channel, codes 
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deriving secrecy from resolvability are more powerful than those deriving secrecy from capacity 
(Proposition 2); we also derive the secrecy capacity region for general broadcast channels with cost 
constraint and for strong secrecy metrics (Theorem 2 and Theorem 3); 

• We further leverage the connection between secrecy and resolvability to revisit various models of 
secure communication in Section VI. We first provide a simple proof of the strong secrecy capacity 
of ergodic fading wireless channels with full channel state information [9], [23] (Proposition 3). We 
then show that achievable rates already known for mixed channels and compound channels [24], 
[25], can be obtained with conceptually simple proofs, and that these results hold under stronger 
secrecy metrics than was previously established (Proposition 4 and Proposition 5). 

• We exploit the general characterization of secrecy capacity to bound the secret-key capacity of 
a general discrete source model for secret-key agreement (Theorem 4). This result is obtained 
by constructing a coding scheme for secret-key agreement from a coding scheme for a wiretap 
channel. The form of the result, which involves conditional entropy instead of mutual information, 
suggests that the fundamental mechanism behind secret-key agreement is not resolvability but rather 
channel intrinsic randomness [26]. Nevertheless, resolvability provides useful insight for secret key 
agreement. The problem of deriving secrecy from intrinsic randomness is beyond the scope of the 
present work and will be analyzed in a forthcoming paper. 

D. Outline 

The remainder of the paper is organized as follows. Section 11 sets the notation used throughout 
the paper and briefly reviews the fundamental concepts and results of information-spectrum informa- 
tion theory. Section III introduces and analyzes several secrecy metrics that can be used to measure 
information-theoretic security. Section IV analyses the fundamental limits of secure communication for 
Shannon's cipher system. Section V, which forms the core of the paper, proves the impossibility of 
achieving strong secrecy capacity with random codes deriving secrecy from capacity for some wiretap 
channels and establishes the secrecy-capacity region of general broadcast channels with confidential 
messages. Section VI presents applications of the general results to wireless channels, mixed channels 
and compound channels, and secret-key agreement, which may be of independent interest. Section Vn 
offers some concluding remarks. The technical details of the proofs are organized into a series of lemmas, 
whose proofs are relegated to the appendices to streamline the presentation. 
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II. Notation and Foundations 

To fix notation for the sequel, consider three random variables X, Y, and Z with sample values x, y, and 
z taking values in alphabets X, y, and Z, respectively. The joint probability distribution is denoted pxYZ, 
and the marginal probability distributions are denoted by px, Py, and pz- Unless mentioned otherwise, 
alphabets are assumed to be abstract alphabets, including countably infinite or continuous alphabets. If 
the alphabets are finite, then the probability distributions correspond to probability mass functions; if the 
alphabets are uncountable, then the probabiUty distributions correspond to probability densities, which 
we assume exist^. The mutual information between X and Y is the random variable^ 

I(X; Y) = log — — — . 

The average of the mutual information random variable is the usual average mutual information, which 
we denote by I(X; Y). For discrete random variables, I(X; Y) has the familiar expression 

I(X; Y) . E„1.(X; Y)] ^ E E «v(x,,) log J^^. 

The conditional mutual information between X and Y given Z and the average conditional mutual 
information are accordingly defined as 

I(X;Y|Z)^log-^^|^^^^^^ and I(X; Y|Z) ^ Exyz[I(X; Y|Z)], 

respectively. Similarly, the entropy and average entropy of X are 

H(X)^log^— and M(X) ^ Ex[H(X)], 

and the conditional entropy and average conditional entropy of X given Y are 

H(X|Y)^log ^ and M(X|Y) ^ Exy[H(X|Y)]. 

The binary entropy function is denoted by : p — > — plogp — {1 — p) log(l — p). All the traditional 
relations between average mutual information and average entropy that result from basic properties of 
joint, marginal, or conditional probability distributions can be shown to hold with probability one for the 
mutual information and entropy random variables. In particular, the chain rules of mutual information 
and entropy hold with probability one. 

^We note that more general situations can be treated with the approach of Pinsker [27]. 
^Unless indicated otherwise, logarithms and exponentials in the paper are taken to base two. 
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In the remainder of the paper, we often measure the similarity of two random variables X E X and 
X' e X in terms of the variational distance between their distributions, defined as"* 

'V{px,Px') = 2 sup |Px[^] - Px'[^]| • 

The variational distance is not as convenient to manipulate as the average mutual information, but we 

provide simple rules for variational distance calculus in Appendix A. 

Given two sequences of arbitrary random variables {X" € ^"}n>i ^^'^ ^ -^"}n>i' characterized 
by a sequence of joint probability distributions {px-'vln^i' the probability distribution of ^I(X"; Y") 
is referred to as the mutual information rate spectrum. In addition, the spectral-inf mutual information 
rate is defined as [7] 

p-hminf -I(X"; Y") = sup |/3 : lim P 

and the spectral-sup mutual information rate is defined as 

p-limsup -I(X"; Y") = inf I a : lim P 

n->oo n I n^oo 

For convenience, we recall that these two quantities, which represent the extreme points of the support 
of the random variable ^I(X"; Y") in the hmit of large n, have an important operational significance for 
point-to-point communication channels. Given a general channel {X, y, {pY"|X"}n^i) with input alphabet 
X, output alphabet y, and transition probabilities {pvix^ln^i^ the spectral-inf mutual information rate 
characterizes the channel capacity, defined as the supremum of reliable communication rates over the 
channel. 

Theorem (Verdu-Han [7], [17]). The channel capacity C of a channel (-^,3^, {pY'»|X"}n^i) is 

C= max p-liminf -I(X";Y''). 

n— >-oo IT' 

The spectral-sup mutual information rate characterizes an upper bound for the channel resolvability, 
defined as the infimum rate of uniform randomness required to reproduce any process at the output of 
the channel with arbitrary precision, measured in terms of variational distance. 

Theorem (Han-Verdu [5], [7]). The channel resolvability S of a channel (-^j iV, {pvix^jn^i) satisfies 

max p-hmsup -I(X";Y''). 

''This general definition of variational distance reduces to Ylxex |Px(x) — Px'(x)| if X is countable. 
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Similarly, given an arbitrary process {X" G X^}n^i, the entropy rate spectrum is the distribution of 
the random variable iH(X"), and the spectral-inf entropy rate is defined as 

p-hminf ^ H(X") = sup |/3 : lim P 

n-^oo n [ n-*-oo 

while the spectral-sup entropy rate is 

p-Umsup ^ H(X'*) = inf |a : lim P 

n-*-oo n y n->oo 

Again, these two quantities have an operational significance for sources of information. Given an arbitrary 
source {X ,{px^}n^i) with alphabet X and symbol sequence probabilities {px"}n^i, the spectral-inf 
entropy rate represents the source intrinsic randomness, that is the maximum rate of uniform randomness 
that can be extracted from it. 

Theorem (Vembu-Verdu [7], [28]). The source intrinsic randomness Si of a source {X ,{px'^}n'^i) is 

5j =p-liminf -H(X''). 

n— >-oo ^ 

The spectral-sup entropy rate has a dual role and characterizes the source resolvability, that is the 
infimum rate of uniform randomness required to simulate it with arbitrary precision, measured in terms 
of variational distance. 

Theorem (Han-Verdu [5], [7]). The source resolvability Sr of as source iX,{px"}n^i) is 

5^ =p-limsup -H(X"). 

n— >-oo ^ 

As we will see, the spectral-sup and spectral-inf mutual information and entropy rates also play 
fundamental roles in the analysis of secure communications, and many results combine these quantities 
in various ways. 

III. Preliminaries: Secrecy Metrics 

Let n eN* and R> 0. Let M. G [1, 2"^] be a random variable with uniform distribution that represents 
a message in a communication scheme. Assume that an eavesdropper has some knowledge about M. 
represented by another random variable Z" G Z^, characterized by the joint probabiUty distribution 
PmZ" • As mentioned in the introduction, message M. is information-theoretically secure if it is statistically 
independent of Z"; however, exact statistical independence between M and Z" is extremely stringent 
and, for tractability, it is convenient to use a slightly weaker measure of secrecy, by which we only 
require M, and Z" to be asymptotically independent as the parameter n tends to infinity. Note that there 



-H(X") < P 
n 



-H(X^) > a 
n 
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is some leeway in the definition of asymptotic independence because one can choose a particular metric 
to measure dependence of M. and Z". For instance, given any distance d for the space of joint probabihty 
distributions on [1, 2"^]] x Z"-, the quantity d{pMZ"',PMPZ" ) could be used as a metric, and asymptotic 
statistical independence then amounts to the condition 

lim (i(pMZ";PMPZ'') = 0. 

n— >oo 

In the following, we specify six reasonable choices for secrecy metrics. The first metric measures statistical 
independence using the KuUback-Leibler divergence: 

Si (mz",PMPZ") = B(pMZ"lbMPZ") = I(M;Z"). 

Note that the secrecy condition lim„^oo §1 (pmz",PmPZ") = is the well-known strong secrecy 
condition [12]. A second metric that is particularly useful is based on the variational distance: 

§2 iPM.Z",PM.PZ") = V(pMZ",mPZ")- 

For any e > 0, the asymptotic independence of M. and Z" can also be measured in terms of the CDF of 
I(M;Z"): 

Sa (mz",mPZ") = P[I(M;Z") > e], 
in which case the secrecy condition 

Ve > lim S3 (pmz",PmPZ") = 

n->oo 

means that the random variable I(M.; Z") converges in probability to zero. Finally, we could also use 
weakened versions of the metrics above by introducing a normalization by a factor of n as 

S4(mz",mpz'.) = ^i^lmz-lbMPz") = ^KM.^z''), 
§5 (mz",mpz'') = ^V(pMZ",mpz''), 

for€>0 S6(mz",mPZ") = P[^I(M;Z") >e]. 

The secrecy condition lim„_^oo §4 {pmZ" , PM.Pz^ ) = is the weak secrecy condition initially introduced 
by Wyner [1]. 

Note that the secrecy conditions^ lim^^oo (pmZ" > PmPZ" ) = may not be equivalent for all i G 
[1,6]]; by establishing an ordering among the previous metrics, we formalize what it means for a metric to 

^The limit should be understood for any e > in the case of metrics §3 and Se- 
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be "stronger" than another. Formally, for i,j G [1,6], we say that Sj is stronger than Sj (or equivalently 
that Sj is weaker than Sj), and we write Sj y Sj if and only if 

lim Si(mz»,PMPZ") = ^ lim Sj(pMZ",mPZ") = 0. 

n-^oo n->oo 

By construction, it is clear that Si y S4, S2 >: S5 and S3 y Se; however, we establish a more precise 
result. 

Proposition 1. The secrecy metrics Si for i e |1,6] are ordered as follows. 

Si ^ §2 ^ S3 t S4 ^ §5 t Se- 

Proof: See Appendix B. ■ 
A direct consequence of Proposition 1 is that any secure communication scheme satisfying the strongest 
secrecy metric Si automatically satisfies the secrecy metrics Si for i e [[2,6]]. Conversely, any secure 
communication scheme that does not satisfy the weakest secrecy metric Sg cannot satisfy any of the 
metrics S^ for i G ]]1, 5]]. Therefore, to establish a coding theorem for a secure communication scheme, 
we can prove achievability for the strongest metric §1 and the converse for the weakest metric Sg- 

Although the ordering in Proposition 1 follows strictly from mathematical properties, the idea that some 
metrics are stronger than others is also meaningful from a cryptographic perspective. One can construct 
examples of communication schemes that present obvious security loopholes while still satisfying the 
weak secrecy metric S4 (see for instance the examples in [4], [23], [29]). It is now accepted that 
information-theoretic results should hold at least under the secrecy metrics^ Si or S2. 

IV. Shannon's Cipher System 

As a first illustration of the connection between secrecy and resolvability, we elaborate on Example 1 
and revisit Shannon's cipher system. We consider the model illustrated in Figure 2, in which a message 
M uniformly distributed in [1,2*^^1 is to be communicated reliably from a transmitter (Alice) to a 
legitimate receiver (Bob) in the presence of an eavesdropper (Eve). Alice and Bob have access to a 
common discrete source of randomness (/C, {pK"}n^i)> characterized by an alphabet /C and a sequence 
of symbol probabihties {pK''}n^i> which is used to encode M. into a codeword Z e Z. Bob's estimate 
of the message using Z and the source is denoted by M. 

*The metrics could be further strengthened by imposing an exponential convergence with n; however, except in the case of 
exponentially information stable channels [13], such as memoryless channels, we were unable to prove general results with this 
additional constraint. 
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Alice 



ENCQDER 



Bob, 



DECODER [*- M 



Eve 



Fig. 2. Shannon's cipher system for a general common source of randomness. 



Definition 1. A (2"^,n) code Cn for Shannon's cipher system consists of 

• an encoding function /„ : [1, 2"^]] x /C" — t- Z that encrypts a message into a codeword; 

• a decoding function gn ■ 2 x /C" |1, 2"^^] that decrypts a codeword into a message. 

The reliability performance of a code C„ is measured in terms of the probability of error 

while its secrecy performance is measured in terms of the secrecy metric^ Si(Cn) — ^i{pM.Z:PM.Pz)- 

Definition 2. A rate R is achievable for secrecy metric Sj with i G [1,6] if there exists a sequence of 
(2"^, n) codes {Cn}n^i such that 

lim Pe(Cn) = and lim Sj(C„) = 0. 

n— >-oo n— >-oo 

The secrecy capacity ci*^ for secrecy metric Sj is 

Cg' = sup{i? : Ris achievable for secrecy metric Si}. 

Tlieorem 1. The secrecy capacity for secrecy metric Sj with i G [2, 6] is the same and is given by 

C^'^ = p-liminf -H(K'*). (1) 
If the source (/C, {pK"}n^i) memoryless, then the secrecy capacity is also the same for metric Si. 



Proof: We first show that all rates below p-Uminf ^H(K") are achievable for secrecy metric S2. Let 

n— >oo 

e,7 > and R = p-liminf ^H(K") — 7. Let Ur be the random variable with uniform distribution 

on [[1,2"'^]1. By [28, Lemma 3], there exists an encoding function /„ : /C" ^ [[liS"-^! such that 
V(p/„(k"))PUh) ^ Cn with lim^^ooCn = 0. A message M, is then encoded as Z = /n(K") © M, 

'We drop the conditioning on C„ in probability distributions when this is clear from the context. 
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where © represents the addition modulo [2"^]. By construction, Bob retrieves M. without error since 
M = Z © /n(K"). Then, note that 

S2(Cn) = V(pMZ,mPz) = IEm [V(pz|m,Pz)] 

)] +v(pu«,^?z) 

^2Em[V(pz|m,Pu«)] 
^ 2e^j, 

where the last equality follows from the definition of Z and the independence of /n(K"^) and M.. Therefore, 
the rate R is achievable and, since 7 can be chosen arbitrarily small, we conclude that 

Cf^ ^ p-liminf -H(K''). (2) 

n— >-oo IT' 

If the source (/C, {pK"}n^i) is ii.d., one can easily modify the proof of [28, Lemma 3] to show that, 
if i? = p-liminf iH(K") - 7, there exists a function : /C" ^ [1,2"^1 and > 0, such that 

n— >oo 

V(p/„(K"))PUr) ^ 2~°'~''^. Following the same steps as in the achievability proof above, we then obtain 
that S2(C„) ^ 2 • 2"°''". Then, [13, Lemma 1] shows that there exists > such that, for n large 
enough Si(Cn) ^ 2-^-'". 

We now prove the converse part of the result. Let R be an achievable rate for secrecy metric Se- There 
exists a sequence of (2"^,n) codes {Cnjn^i such that lim„^oo Pel^n) = and lim„^ooS6(Cn) = 0. 
For every n G N*, and with probability one, we have 

-H(M) = -H(M|Z) + -I(M;Z) 
n n n 

= -I(M; K"|Z) + -H(M|ZK") + -I(M; Z) 
n n n 

= -WiyC") - -H(K"|MZ) - -1{YJ';Z) + -HfMIZK") + -I(M;Z). 
n n n n n 

Since R = p-liminf ^H(M), p-liminf iH(K"|MZ) ^ 0, and p-hminf ^I(K"; Z) ^ 0, we obtain 

n— >-oo n— >-oo n— >oo 

R ^ p-liminf -H(K'") + p-limsup -H(M|ZK) + p-limsup -I(M; Z). 

n— >-oo n— >-oo 



By assumption, p-limsup ^I(M.;Z) = since lim„_^oo §6(^0) = 0. The Verdu-Han Lemma [7], [17] 

n— >-oo 

also guarantees that p-hmsup ^H(M|ZK") = 0; hence, we conclude that 

Cf^ ^ p-liminf -H(K"). (3) 
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Fig. 3. Broadcast channel with confidential messages. 



Combining (2) and (3) with Proposition 1, we conclude that, for each i G p, 6]], cf' = p-liminf ^H(K"). 

n— >-oo 

If the source (/C, {pK"}n^i) is i.i.d., then for each i G [1, 6], ci*^ = p-Uminf ^H(K"). ■ 

n— >oo 

The fact that secrecy capacity is identical for all metrics Sj with z G |2, 6] suggests that asymptotic 
statistical independence is indeed a fundamental measure of secrecy. 

Note that the coding scheme used in Theorem 1 extracts the source intrinsic randomness of (/C, {pk" }n^i) 
to protect the message with a one-time pad. Nevertheless, the message is kept secret from the eavesdropper 
because the encoder exploits the randomness of the source to control the distribution of the eavesdropper's 
observation; hence, the coding mechanism for secure communication is closer to channel resolvability, 
which we confirm in the next section. 

V. Secrecy from Resolvability over Noisy Channels 

We now turn our attention to the problem of secure communication over noisy channels. We consider 
a broadcast chaimel with confidential messages {X ,Z ^{Wyn2n\yn)n^\) characterized by an input 
alphabet X, two output alphabets y and Z, and a sequence of transition probabilities {WV»z"|X''}n^i- 
The chaimels {X ,y ^{Wyn\xn\n^\) and {X,Z,{Wz_n\xn}n^i) obtained from the marginals are called 
the main channel and the eavesdropper's channel, respectively. The inputs to the channels are also 
subject to cost constraint P G M^; specifically, there exists a sequence of cost functions {c„}„^i with 
c„ : X'^ — > ]R_|_, such that any sequence x" G Af" transmitted through the channel should satisfy 
^c„(x") ^ P. Following standard practice, the transmitter is named Alice, the receiver observing output 
Y is named Bob, and the receiver observing output Z is named Eve. As illustrated in Figure 3, Alice 
wishes to transmit a common message Mq to both Bob and Eve and an individual message Mi for Bob 
alone, viewing Eve as an eavesdropper for message Mi. Bob's estimates of the messages are denoted 
by Mo and Mi while Eve's estimate is denoted by Mq. 
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Definition 3. A (2'^^ ,2'"^\n) wiretap code Cn consists of 

• a common message set Mq = \1, 2"^-^"]; 

• an individual message set Aii = [[l,2"'^i]]; 

• an auxiliary message set M.'^ = [1, 2**^!], with R[ > 0,^ which is used to randomize the transmission 
of individual messages; 

• a source of local randomness (7?.,^?r), which can be used to further randomize the encoding process 
and is only known to Alice; 

• an encoding function fn Mo x Mi x M'l xTZ ^ X'^, such that 



• a decoding function '-y^ ^ Mq x M\ x A^'^; 

• a decoding function hn : 2" — > Mq. 

The auxiliary message is denoted by M.'^^. All messages Mo,M.i,M'^ are assumed to be uniformly 
distributed in their respective sets. The size of the auxiliary message set and the source of local randomness 
(7?.,Pr) can be optimized as part of the code design, and the eavesdropper is assumed to know the code 
Cn, which includes the statistics pr of the source of local randomness. In the remainder of the paper, 
we clearly identify the channel inputs and outputs obtained when using a code Cn by introducing a bar 
in the notation of the corresponding random variables. For instance, the random variable representing a 
codeword chosen in C„ is denoted X", those representing the corresponding channel outputs are denoted 



and Z"-, and the joint distribution between Mq, Mi, X", Y", Z" is 

V(mo,mi,x",y",z") eMoxMiX X" x y" x ^" 

PMoMiX"Y''Z"(»^o,?^^i,a;",y",2;'') = VFYnzn|xn(2/", 2"|aj")px"|MoMi(a^''l"^o, m-i) 

PMo("^o)mi("^i) ■ (4) 
The reliability of a code C„ is measured in terms of the average probability of error 



while its secrecy is measured in terms of the secrecy metric Si(C„) = Si(PM.iZ'»'PMiPz") ^ ^ 6]. 
^Unlike Rq and which are fixed parameters, we allow to vary with n. 



V(mo,mi,mi,r) G A^o x Mi x M'l x U -Cn{fn{mo,mi,m'i,r 



Pe(C„) ^ P (Mo, Ml, M;) ^ (Mo, Ml, M'l) or Mo ^ Mq C 



■n 
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Definition 4. A rate pair {Rq,Ri) is achievable for secrecy metric Sj if there exists a sequence of 
(2niio 2«-f^i,n) codes {Cnjn^i such that 

lim Pe(Cn) = and lim Sj(C„) = 0. 

n— >oo n->oo 

The secrecy-capacity region TZbcc for secrecy metric Sj is 

TZgcc — closure {{{Rq, Ri) : {Rq, Ri) is achievable for secrecy metric Sj}) ; 
the secrecy capacity for secrecy metric Si is 

Cg' = sup{i?i : (0, Ri) is achievable for secrecy metric Sj}. 

Note that our definition of a wiretap code explicitly introduces the randomness used in the encoding 
process. The randomness is split between an auxiliary message with uniform distribution and a source 
of local randomness and, in addition, we require the auxiliary message to be decoded by the legitimate 
receiver. Since the source of local randomness can be arbitrarily chosen, our definition incurs no loss of 
generality; however, this allows us to explicitly define the class of capacity-based wiretap codes, which 
is implicitly used in [1], [2]. 

Definition 5. A (2^^ ,2^^\n) capacity-based wiretap code Cn is a (2"-^°, 2"^^ , n) wiretap code such 
that : 

• the auxiliary message rate is R'^ = Cg — e„, where Ce is the eavesdropper's channel capacity and 
{cnln^i is such that lim„-^oo = and lim^^oo ^ny/n = oo; 

• there exists an additional decoding function h'^ : x A^i — > M.\, which allows the eavesdropper 
to estimate the auxiliary message M.'^ from the observation of TP' and Mi. 

We let M'l denote Eve's estimate of M.'^. The reliabiUty of a capacity-based wiretap code C„ is then 
measured in terms of the modified average probability of error 

p:(C,) ^ P [(Mo, Ml, M'l) ^ (Mo, Ml, M'^) or (Mo, M'l) + (Mo, m;)|c„ 

Definition 6. A rate pair (Rq, Ri) is achievable for secrecy metric Sj with capacity-based wiretap codes 
if there exists a sequence of (2'*^°, 2"^% n) capacity-based wiretap codes {Cn}n^i such that 

lim P*(Cn) = and lim Si(C„) = 0. 

n->oo n->oo 

The constraint lim^^oo Pe(Cn) = ensures that, given knowledge of Z" and Mi, the eavesdropper 
could rehably decode the auxiliary message M'^. Nevertheless, since the eavesdropper does not have 
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access to the message Mi, this property is solely used to impose structure on the code. The denomination 
"capacity-based code" is used because the set of codewords associated to a known pair of messages 
(Mo, Ml), which can be thought of as a sub-code of rate R[ = Ce — e„, stems from a sequence 
of capacity-achieving codes for the eavesdropper's channel. This property, which is formalized in [30, 
Theorem 1], is implicitly used in most works that show the existence of wiretap codes achieving secrecy 
rates for metric §4. 

Remark 1. Since Pe(Cn) only depends on the marginals {VFY"|X"}n^i Sj(Cn) only depends on the 
marginals {lFzn|X"}n^i. the performance of a wiretap code only depends on the marginals; however, 
this property is lost with capacity-based wiretap codes because ¥*^{Cn) depends on {M^Y"Z''|X''}n^i- 

Remark 2. Csiszdr and Korner [2 ] analyze the fundamental limits of secure communication more pre- 
cisely by studying the rate-equivocation region (Rq, Ri, R^), where Re ^ Ri represents the equivocation- 
rate ^H(Mi|Z") of the eavesdropper about the individual message. Unlike the rates Rq and Ri, the 
notion of equivocation depends on the secrecy metric considered; therefore, we restrict ourselves to the 
special case of full secrecy rates R\ = Rg, for which we can leverage the result of Proposition 1. 

In the absence of a common message {Rq = 0), a broadcast channel with confidential messages is 
concisely called a wiretap channel, and a (l,2"^i,n) code is simply denoted as a (2"-f^Sn) code. 

A. Capacity-Based Wiretap Codes May Not Achieve Strong Secrecy 

All the analyzes of wiretap channel models based on capacity-based wiretap codes derive secrecy for 
metric S4. Additional modifications of the codes based, for instance, on privacy ampUfication [12], [14] are 
required to achieve secrecy for stronger metrics. In this section, we show that this may be a fundamental 
hmitation of capacity-based wiretap codes by proving that sequences of random capacity-based wiretap 
codes that achieve the weak secrecy capacity cannot achieve the strong secrecy capacity. 

Specifically, we consider a particular wiretap channel, in which the main channel and the eavesdropper's 
channel are both binary symmetric channels with respective cross-over probability Si and S2, such that 
< 61 < 62 < l- We further assume that no cost constraint is imposed (Vx" G Af" c„(x'^) = n 
and P = 1) and no source of local randomness is available. Information-theoretic proofs using random 
codes [1] or polar codes [31] show that the absence of source of local randomness incurs no loss of 
optimality for this channel. 
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Proposition 2. Let {C„}„^i be a sequence of {T^^^n) random capacity-based wiretap codes, obtained 
by generating codeword symbols independently and uniformly at random. Let the rate of the auxiliary 
message be such that i?' = 1 — Hfe {82) — e„ and R + R' = 1 — Hi {5i) — and assume there is no 
source of local randomness. Then, there exists ri,a > 0, such that, for n sufficiently large. 



S2(C„) > ri, KiCn) ^ 2-2^"" and S^Cn) ^ 2e, 



>l-2- 



i.e., with high probability over the random code ensemble, a sequence of capacity-based random codes 
achieves the weak secrecy capacity but does not achieve the strong secrecy capacity. 

Proof: See Appendix C ■ 
The result of Proposition 2 generalizes to symmetric channels [32] and we conjecture that it also 
holds for asymmetric channels, as well as non-random codes. Despite its lack of generality, Proposition 2 
shows that a random construction with capacity-based wiretap codes is not powerful enough to prove 
strong secrecy results, which suggests exploiting a more powerful mechanism to ensure secrecy. In the 
remainder of the paper, we derive secrecy from resolvability and show that such codes do not suffer from 
the limitations of capacity-based wiretap codes. 

Remark 3. If the main channel is noiseless. Proposition 2 can be strengthened to prove that no capacity- 
based wiretap code (including non-random codes) achieves secrecy capacity for metrics S2 and Si. This 
fact was noted in [31 ] for metric Si with a different argument based on results for finite blocklength 
channel coding [33]. 

B. General Broadcast Channels with Confidential Messages and Cost Constraint 

In this section, we establish the secrecy-capacity region of a general broadcast channel with confidential 
messages for secrecy metrics Sj with i G [2,6]]; the alphabets and transition probabilities of the channel 
{VFY"Z"|X"}n^i ^£ arbitrary, so that the model includes continuous channels and channels with memory. 
Following the conclusions drawn from Proposition 2, we analyze codes that are more powerful than 
capacity-based wiretap codes and whose secrecy is tied to the notion of resolvabiUty. 

Theorem 2. The secrecy capacity region of a broadcast channel (^X,y,Z,{WY^Z'^\X'^}n^i) with con- 
fidential messages and cost constraint P is the same for secrecy metrics Sj with i G j]2,6| and is given 
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by 



{Ro,Ri) : 

O^Ro^ min ( p-Uminf -I(U"; Y"),p-liminf -I(U"; Z' 
O^Ri^ p-Uminf -^V^; Y"|U") - p-limsup -I(V"; Z"|U 

n—^oo IT' n— >-oo IT' 



(5) 



where 



V = { {U^'V"X"}^>^ : Vn G N* 



an^/P[ic„(X")^P] =1 

Notice that the form of the secrecy capacity region is the natural generalization of that obtained for 
memoryless channels in [2, Corollary 1]; however, the main channel statistics affect the secure rate Ri 
through their "worst realization" (p-liminf ^1(V"; Y"|U")) while the eavesdropper's channel statistics 

n— >cxD 

affect it through their "best realization" (p-limsup ^liy^; Z"|U")). Intuitively, as illustrated in Figure 4, 

n— >-oo 

this occurs because the worst case for secure communication is when the main chaimel conveys the 
smallest information rate to the legitimate receiver while the eavesdropper's chaimel leaks the largest 
information rate to the eavesdropper. It will be apparent in the proof that this asymmetry, which disappears 
in the case of memoryless chaimels, happens because the coding mechanisms used to ensure reliability 
and secrecy are different. 



Umit distribution of il(V"; Z"|U") 



Umit distribution of il(V"; Y"|U") 





p-liminf il(V";Z"|U") p-limsup il(V"; Z"|U") p-liminf il(V"; Y"|U") p-limsup il(V"; Y"|U" 



Fig. 4. Illustration of secure rates in Theorem 2. 



Without a common message {Rq = 0), we obtain in a similar way the secrecy capacity of a general 
wiretap chaimel^. 



'The result in Corollary 1 was already obtained by Hayashi in [15, Lemma 4 and Lemma 5] without cost constraint. Our 
proof technique is different from that of Hayashi, which is based on a non-asymptotic analysis. 
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Corollary 1. The secrecy capacity of a wiretap channel (-Y, y, Z, {VFY"Z''|X"}n^i) "^ith cost constraint 
P is identical for secrecy metrics S, with i E [2, 6] and is given by 

Cs = max ( p-Uminf -I(V"; Y'') - p-limsup -IfV; Z'^) ) , (6) 



where 



„ . V" -> X" Y"-T'^ forms a Markov chain 

and^{lcn{X^)^P\=\ 



> . 



Remark 4. The general achievability results of Theorem 2 and Corollary 1 are established for metric 
§2. We require additional assumptions on the channel statistics to establish secrecy for metric Si, see 
Remark 5. 

Proof of Theorem 2: We start with the achievability part of the proof, for which we create a codebook 
by combining superposition coding and binning schemes. Let n G N* and e,^, Rq, Ri, R[ > 0. Define 
Mo = [2"^«], Ml = [2"^i] and M[ = [2"^'!]. Let W be an arbitrary alphabet and fix a distribution 
Pu^ on W". Fix a conditional distribution Px^lU" on x Z^" such that P[^c„(X'*) ^ P] = L Let 
U", X", Y", Z" be the random variables with joint distribution 

VK, x", y", z") G X -Y" X X 

• Code generation: Randomly generate Mq sequences e with A; e [1, Mo] according to pw^. 
For each A; G [l,Mol, generate MiM[ sequence x^^^ G -Y" with (/,m) G [l,Mil x [1,M(1 
according to Px''|U''=ug- We denote by C„ the random random variable representing the generated 
code and by C„ one of its realizations. 

• Encoding: To transmit a message pair {k,l) G [[l,Mo]] x [[l,Mi]], Alice generates an auxiliary 
message m uniformly at random in [1, M(| and sends the codeword 'X-]^i^ through the channel. 

• Bob's decoding: Define the sets 

A r 1 py" I u" 1 1 

[ n PY"(y") n J 

rp ^ {u^, x", y") eU^xX^xy^:- log ^>^"|X"U"V^ I ' ' ^ _ MiM[ + 7 L 
l_ n PY"|U"(y 1^^ ) n J 

Upon observing y", Bob decodes k as the received common message if is the unique sequence 
in Cn such that (u^,y") G Ti', otherwise, a random message is chosen. Similarly, he decodes I as 
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the received individual message and m as the received auxiliary message if there exists a unique 
codeword x^^^ such that i'^k^^kim'y") ^ otherwise, random messages are chosen. 
• Eve's decoding: Define the set 

7? ^ ( (u" , z") G X : - log ^^-m"!^]'^"^ ^ i log Mo + 7 ) 

Upon observing z", Eve decodes k as the received common message if is the unique sequence 

such that (u^,z") G T^^; otherwise, a random message is chosen. 
The following lemmas, whose proofs are relegated to Appendix D, provide sufficient conditions to 
guarantee reliabihty and secrecy. 



lim E[Pe(C„)] ^ e. 



Lemma 1 (Reliability conditions). 

Ro ^ min (p-liminf ^I(U"; Y") - 27,p-liminf ^I(U"; Z'*) - 27 J 
Ri + R[ ^ p-liminf il(X"; Y"|U") - 27, 

n— >-oo 

Lemma 2 (Secrecy from resolvability condition). 

i?; ^ p-limsup -I(X"; Z"|U") + 27 ^ lim E[S2(Cn)] ^ e. 
Combining Lemma 1 and Lemma 2, we obtain 

Ro ^ min (p-liminf ll(U- Y") - 27,p-liminf il(U";Z") - 27) [ lim„_oo M[Pe(C„)] ^ e 
Ri ^ p-liminf ^I(X'*;Y"|m) - p-limsup ^I(X"; Z"|U'^) - 47, \ lim„^oo IE[S2(C„)] ^ e 

Using Markov's inequahty and the union bound, we can prove there exists at least one sequence of 

^2niio 2"-'^i,n) codes {Cn}n^i such that lim„_>ooPe(C„) ^ 3e and lim„^oo S2(Cn) ^ 3e. Since e and 7 
can be chosen arbitrarily small, we conclude that 

{Rq,Ri) : 



U 



O^Ro^ min (p-liminf -I(U'';Y"), p-liminf -I(U"; Z") ) , 
O^Ri^ p-liminf -I(X'";Y"|U") - p-limsup -I(X"; Z"|U'") 



) c 7^^,^' (8) 



where 

U" ^ X" ^ Y"Z" forms a Markov chain 

andP[ic„(X") ^P] =1 



> . 



Finally, note that the source of local randomness {Tl,p^) can be used to prefix an arbitrary channel 
(V, X, {px^ivln^i) to the broadcast channel (Af, y, Z, {Wv^z^ix^jn^i)- By applying the proof above 
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to the concatenated channel (y,y,Z, {pY'»Z"|V"}n^i)> we conclude that the region given in Theorem 2 
is included in the capacity region TZ^cq. 

We now turn to the converse part of the proof. Consider a sequence of codes {C„}„^i achieving the 
rate pair {Rq,Ri) for secrecy metric Sg- For ^ G let U" denote the choice of a common message 
uniformly at random in [1,2"^"]] and let denote the choice of an individual message uniformly at 
random in [1, 2"^^]]. Let Y" and Z" denote the channel outputs corresponding to the transmission of the 
message pair (U",W"). As shown in Appendix E, the following lemmas hold. 

Lemma 3. 

" i?o ^ min ( p-hminf ^l(U"; Y"),p-hminf ^l(U'*;Z'*) 
lim Pe(Cn) = 0^ { 

n->-oo 



Ri ^ p-hminf il(W"; Y'^IU"). 



Lemma 4. 



lim„_,ooPe(C„) =0 1 

^ p-hmsup -I(W^; Z |U ) = 0. 
lim„_,ooS6(Cn) = n-^oo n 

Therefore, combining Lemma 3 and Lemma 4, it must hold that 

Ro ^ min (p-hminf - 1 (U'*;Y"), p-hminf -l(U'*;Z'*) ) 

Ri ^ p-liminf -l(\r';Y"|U") - p-limsup -l(W"; Z"|U") . 

n— >oo ^ n—^oo 

Note that, by assumption, U"W" ^ X" ^ Y"Z" forms a Markov chain. Define V" = (U", W"), which 
is such that U'^ ^ V" ^ X" ^ y"Z" forms a Markov chain. With probability one, we have 

I(W'^;Y"|U") = I(V";Y'^|U") and l(Vr;Z"|U'^) = l(V"; Z^jU'^); 

therefore, an achievable pair {Ro,Ri) must satisfy 

Ro ^ min (p-liminf - 1 (U" p-liminf -l(U";Z") J , 

and Ri ^ p-liminf -l(V"; Y"|U") - p-limsup -l(V"'; Z"|U") , 

n— >-oo ^ n— >-oo ^ 

where U" ^ V" ^ X" ^ Y"Z" forms a Markov chain, Py'-z-IX" = W^ynz-iX", and P [^Cn(X") ^ P] = 
1. Taking the union over all possible processes {U"V"X"}„^i gives the desired outer bound for the 
secrecy capacity region 7?.bcc- 

Since the outer bound for 7?.bcc and the iimer bound for 7?.bcc niatch, we conclude using Proposition 1 
that the secrecy capacity is the same for all metrics z G [2, 6]. ■ 
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A few comments regarding the proof of Theorem 2 are now in order. First, the achievability part of 
the proof is based on an explicit operational interpretation of secrecy in terms of channel resolvability; 
in Lemma 2, codes are constructed so that, for a given message Mq, the probability distribution induced 
at the eavesdropper's channel output by all messages Mi is the same. Second, the mechanics of the 
proof are fundamentally different from the standard approach of Wyner [1] and Csiszar and Korner [2]. 
The existence of a sequence of codes simultaneously satisfying the reliabihty and secrecy conditions 
is obtained by handling the constraints separately, as illustrated by the separate results of Lemma 1 
and Lemma 2. This contrasts with the approach of [1], [2], in which the two constraints are handled 
somewhat simultaneously by using capacity-based wiretap codes. As should be clear from the condition 
R'l > p-Umsup ^I(X";Z"|U") obtained in Lemma 2, the codes constructed are not capacity-based 

n—^oo 

wiretap codes, for which the condition would read R[ < p-liminf ^I(X"^; Z"|U"); essentially, channel 

n—^oo 

resolvability allows us to analyze the behavior of codes operating at rates beyond the capacity of the 
eavesdropper's channel. 

Remark 5. A closer look at the proof of Theorem 2 shows that we could strengthen the secrecy metric 
and prove that S2(Cn) decays exponentially fast with n provided the quantity 



U"X"Z" 



-I(X";Z"|U'^) > -logM( + e 
n n 



decays exponentially fast with n for any e > 0. We do not explore this issue further for arbitrary 
channels but we analyze it more precisely in the next section for memoryless channels. 

We conclude by noting that the invariance of the secrecy capacity region with respect to the metrics 
Si for i G [2, 6] suggests that asymptotic statistical independence is indeed a fundamental measure of 
secrecy because the fundamental limits of secure communication seem to remain unchanged no matter 
how statistical independence is measured; nevertheless, we emphasize again that practical coding schemes 
should be designed to provide the strongest level of secrecy. 

C. Memoryless Broadcast Channels with Additive Cost Constraint 

We now consider memoryless channels (not necessarily discrete) with an additive cost constraint. This 
is a special case of the general model, in which the transition probabilities factor as 

n 
i=l 

'"This property is called exponential information stability in [13]. 
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and the cost constraint satisfies 

n 

Vx" G A"" c„(x'^) = ^c(xi) for some cost function c: X ^ M+. 

For this special class of channels and constraints and under mild conditions, we can strengthen the results 
of Section V-B and establish the secrecy capacity region for metric Si. 

Theorem 3. The secrecy-capacity region of a memoryless broadcast channel {X,y,Z,WYz\x) ^i^h 
confidential messages and additive cost constraint P for secrecy metric Sj with i E [2,4] is 

{ {Ro,Ri) ■■ 

^ i?o ^ mill (I(U; Y), I(U; Z)) } , (9) 
O^Ri^ I(V;Y|U) -I(V;Z|U) 



^BCC — 

(uvx)ep 



where 

V = {(UVX) : U ^ V ^ X ^ YZ forms a Markov chain and E[c(X)] ^ P} 

If the rates on the boundary of Tl^cc are obtained for some random variables UVXYZ such that the 
moment generating functions of I(V; Z|U) and c(X) converge unconditionally in a neighborhood of 
and are differentiate at 0, then 7?.bcc is also the secrecy-capacity region for Si. 

In the absence of a common message {Rq = 0), we obtain in a similar way the following result. 

Corollary 2. The secrecy capacity of a memoryless wiretap channel {X ,y, Z,pyz\x) for secrecy met- 
ric Sj with i G p, 4| and additive cost constraint P is 

C7,= max (I(V;Y)-I(V;Z)), 

( V A j G / 

where V = {(VX) : V ^ X ^ YT. forms a Markov chain and E[c(X)] ^ P}. If the random variables 
VXYZ maximizing Cg are such that the moment generating functions of I(V; Z) and c(X) converge 
unconditionally in a neighborhood of and are differentiate at 0, then Cg is also the secrecy capacity 
for Si. 

Remark 6. For general memoryless channels, the weakest metric for which we show Theorem 3 and 
Corollary 2 hold is metric §4. As discussed in the proof of Theorem 3, this can be weakened to metric 
Se for discrete memoryless channels. 

Remark 7. The conditions that yield TZgcc ond Cs for metric Si are sufficient conditions required to 
obtain exponential upper bounds when applying Chernov bounds. These conditions are not too restrictive 
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and are automatically satisfied for discrete memoryless channels and for Gaussian channels with additive 
power constraint. 

Corollary 2 was already obtained for discrete memoryless channels by Csiszar [13] and Maurer and 
Wolf [12] with different tools. Csiszar's approach uses graph-coloring techniques while Maurer and Wolf's 
approach exploits privacy amphfication with extractors. Theorem 3 for discrete memoryless channels 
without cost constraint was also obtained independently in [34] using privacy amphfication. 

Proof of Theorem 3: For discrete memoryless channels, the converse part for secrecy metric Se 
follows from Theorem 2 without resorting to Fano's inequality. Following [7, Theorem 3.5.2], one can 
show that 

p-liminf ^ I(U"; Y") ^ liminf ^I(U"; Y"), p-liminf ^ I(U"; Z") ^ liminf ^I(U"; Z"), 

n-5>oo n n^oo n n-5>oo n n^oo n 

p-hminf -I(V";Y"|U'") ^ liminf -I(V"; Y'^jU"), p-hmsup -I(V"; Z"|U'") ^ limsup -^V"; Z"|U"). 
Hence, any achievable pair {Ro,Ri) must satisfy 

O^Ro^ min (liminf -I(U"; Y"), liminf -I(U"; Z") ) , 

\ n— >^oo n n^oo n J 

O^Ri^ liminf f-I(V"; Y"|U") - -I(V"; Z"|U")^ , 

n^oo yn n J 

and, for any e ^ 0, we have for n sufficiently large: 

^ i?o ^ min f -I(U"; Y"), -I(U'^; Z")^ + e, 
\n n ) 

^ i?i ^ (^^I(V";Y'*|U") - ^I(V";Z"|U'^)^ + e. 

Setting Mo = V" and Mi = U", we obtain the same n-letter upper bound as in [2, Section 5]; therefore, 
the same single-letterization procedure can be applied, which yields the desired result. If the channel 
alphabets are not discrete, the converse in [2, Section 5] holds for secrecy metric S4. 

The achievability of the secrecy-capacity region in Theorem 3 for secrecy metric §2 and without cost 
constraint (Vx G X c(x) = 1 and P = 1) can be directly obtained by substituting appropriate random 
processes in the general expression of Theorem 2. It suffices to choose i.i.d. processes {U"V'*X'*}„^i 
such that, for all n ^ 1 and for all (u,v,x) G W x V x ^, pu„ v„x„ (u-, v, x) = Px|v(^|v)pvu(v,u); 
Khintchin's law of large numbers then guarantees that 

p-liminf -I(U";Y") = I(U; Y), p-liminf -I(U";Z") = I(U;Z), 

n— >-oo n— >-oo ^ 

p-liminf -I(V"; Y"|U") = I(V; Y|U), p-limsup -I(V";Z"|U") = I(V;Z|U), 

n— >oo n— >oo ^ 
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which yields the desired result; however, additional work is needed to deal with the cost constraint and 
to obtain secrecy under metric Si. Details are provided in Appendix F. ■ 

Remark 8. In the proof of Theorem 3, we actually establish a stronger result than the one stated. If the 
conditions for the moment generating functions o/I(V; Z|U) and c(X) are satisfied, we show that Si(C„) 
vanishes exponentially fast with n. 

Remark 9. Consider a Gaussian wiretap channel with power constraint P, for which Wy\x ^ -^(0, cr^) 
and VF^ix ~ A/'(0, cTg) with ^ cr^. Substituting V = and X ~ A/'(0, P) in Corollary 2, we obtain 
that all rates R, such that 

^<^°<^ + ^)-^'°<' + |) 

are achievable secrecy rates for metric Si. Together with the converse proof for metric S4 in [8], this 
establishes the strong secrecy capacity of the Gaussian wiretap channel. 

VI. Applications 

In this section, we illustrate the usefulness of deriving secrecy from resolvability by considering several 
problems in which the derivation of achievable secrecy rates is tremendously simplified. In particular, 
results for wireless channels, mixed wiretap channels and compound wiretap channels come almost "for 
free". For clarity, we only consider cases in which the common message rate is zero (Rq = 0). 

A. Ergodic Wireless Channels with Full Channel State Information 

We consider the situation in which Alice and Bob communicate over an ergodic fading wiretap channel 
and have access to the instantaneous fading gains for both the main channel and the eavesdropper channel. 
Specifically, at each time A; ^ 1, the relationships between input and outputs are given by 

where {H^,fe}fe^i, {He,fe}fe^i are fading gains known to all parties and {'N^^k}k^i, {^e,k}k^i are i.i.d. 
complex Gaussian zero-mean noise processes with respective variance cr^ and a^. In addition, channel 
inputs are subject to the long-term power constraint ^ Ylk=i ^ [^fe] ^ ^^ 
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Proposition 3. The secrecy capacity of the ergodic wireless channel with full channel state information 
for secrecy metric Si is 



Co = max E 



log 1 + 



log 1 + 



|He| 7(H^,He 



(10) 



where the maximization is over all power allocation functions 7 : — > M"*" such f/ia/ E[7(H^, He) ^ P]. 

Proposition 3 states the strong secrecy capacity of wireless channels with full channel state information. 
This result has already been established in [23] with a completely different approach; deriving secrecy 
from resolvability and leveraging Corollary 2 provides a much simpler proof, which can be generalized 
to include the effect of imperfect channel state information [35]. 

Sketch of proof: We only sketch the achievabihty part of the proof; the converse for secrecy 
metric S4 is established in [9]. Because the channel gains are instantaneously known to all parties, the 
ergodic wireless channel can be demultiplexed into a set of independent Gaussian wiretap channels, each 
characterized by a specific reaUzation (Kr„,He) of the channel gains and subject to a power constraint 
^{\Vm,\ie). According to Remark 9, the secrecy capacity of each channel for metric Si is 



log 1 + 



|h^| 7(Km,He 



log 1 + 



|He| 7(Km,He 



Hence, using the ergodicity of the channel, we conclude that all the rates R such that 



^ i? < max E 

7 



log 1 + 



iHml 7(H^,He 



- log 1 + 



|He| 7(H^,He 



are achievable for metric Si, where 7 : — >• satisfies E[7(Hr„, He) ^ P\. 



B. Mixed and Compound Channels 

As another application of deriving secrecy from resolvability, we analyze the case of mixed and 
compound wiretap channels. These models have practical relevance since they allow one to analyze 
situations in which the channel is imperfectly known to the transmitter, either because the channel 
estimation mechanism is imperfect or because the channel is partially controlled by the eavesdropper. 

Let A; G N* and let {oik}k<^ii^K\ ^ be such that Ylik=i = 1- Consider K wiretap channels 

^Af,3^, Z, {Wyjz^IX"}^^^ iork G {l, Kj. The mixed wiretap channel is the channel 3^, -Z, Wy"Z"|X") 
whose transition probabilities satisfy 

K 
k=l 
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Proposition 4. The secrecy capacity of a mixed wiretap channel for secrecy metrics Sj with i e [2, 6] is 
max ( min p-Uminf -KV^; YI?) - max p-limsup -KV"; Z^) ) , (11) 



where 

{V"X"}„>i :VnGN* VfcG [1,^1 



■p 4 



V" X" Y^Z^ forms a Markov chain 
ani/P[^c„(X") ^ P] = 1 



> . 



Proof. Using [7, Lemma 1.4.2], we obtain 



p-liminf -I(V"; Y") = min ( p-liminf -I(V"; YI? 

n-).oo n k^\\,K\ V n-J-oo « 

p-limsup -I(V";Z") = max ( p-limsup -I(V"; Z^) 

The result follows by substituting these equahties in Corollary 1. ■ 
Note that, for all i e |l,6l, we have (i^MZ- > PMi^Z" ) ^ Ef=i «feSi(PMZj>PMPz^)- Therefore, a 
code ensuring secrecy for the mixed wiretap chaimel may not guarantee secrecy over each individual 
wiretap channel. If one wants to ensure secrecy over all possible K chaimels, one must consider a 
compound wiretap channel, in which the transmitter has no knowledge (even statistical knowledge) 
of which chaimel in the set is used for transmission; however, to avoid uimecessary mathematical 
complications, we assume that receivers can estimate chaimel statistics perfectly and always know from 
which channel they obtain observations. For every channel k G [[l,-?^!, the performance of a code Cn 
is measured in terms of the average probability of error P^*'(C„) and in terms of the secrecy metric 
Sf'(C„) = Si(PMZ^5?'MPz^)' notion of achievable rate is accordingly modified as follows. 

Definition 7. A rate R is achievable over a compound wiretap channel for secrecy metric if there 
exists a sequence of (2"^^,n) codes {Cn}n^i such that 

VA; G [1, K\ lim (C„) = Q and lim (C„) = 0. 

n— >-oo n— >-oo 

Unlike the mixed wiretap channel, there is no distribution associated to the choice of the channel in the 
set; secrecy and reliability must be guaranteed for all channels in the set, not just the "averaged channel". 

Proposition 5. The secrecy capacity of a compound wiretap channel with cost constraint P is the same 

for secrecy metrics Sj with i G [2, 6] and is given by 

max ( min p-liminf -I(V"; YI?) - max p-limsup -I(V"; Z^) ) , (12) 
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where 

yn ^yn ^ Y^Z^ forms a Markov chain 



> . 



Proof: We start with the achievabihty part of the proof, which is similar to that of Theorem 2. Let 
n G N* and e,^,Ri,R[ > 0. Define Mi = [2"^^] and M[ = [2'^^i]. Fix a distribution pxn on -Y" 
such that ¥[^Cn(X^) ^ P] = 1. Let X", {Yk}keli,iq' m}keli,iq be the random variables with joint 
distribution 

VA; G [1, Kj V(x", y", z") G -Y" x x 

;^X"Y^z^(x^y^z'^) ^ VFYjz^|X"(y",^"k")px"(x") . 

• Code generation: Randomly generate MiM[ sequence G A"** with (Z,m) G |l,Mi] x |1,M(] 
according to px"- We denote by C„ the random random variable representing the generated code 
and by C„ one of its realizations. 

• Encoding: To transmit a message I G [l,Mi], Alice generates an auxiliary message m uniformly 
at random in [1, M{] and transmits the codeword xf^ through the channel. 

• Bob's decoding for channel k G [[l,i<r]: Define the set 

V ^ eX^x y]: :-log -logMiM +7 . 

Upon observing y^, Bob decodes / as the received individual message and m as the received 
auxiliary message if there exists a unique codeword x^ such that (xJJ^,^^) G T"", otherwise, 
random messages are chosen. 

The following lemmas provide sufficient conditions to guarantee reliabiUty and secrecy. Their proofs are 
similar to those provided in Appendix D and are omitted. 

Lenuna 5 (Reliability conditions). For each k G 

Ri + i?; ^ p-liminf -I(X"; Y^) - 27 lim E[Pf (Cn)] ^ e. 

Lemma 6 (Secrecy from resolvability condition). For each k G 

i?; ^ p-limsup -I(X"; ZD + 2j ^ lim E [S'*'(Cn)l ^ e. 
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Using Lemma 5 and 6, we obtain 

Ri ^ min p-liminf -I(X"; Y2) - max p-Uminf -I(X"; Z^) - 47 



lim„^ooK[P<'=>(C„)] ^ e 
lim„^ooK[S^'='(C„)] ^ e 

Using Markov's inequality and the union bound, we can show there exists at least one sequence of 
(2"-f^i , n) codes {C„}n^i such that, for all A; G [1, K], lim„_,oo ^ {K+l)e and lim„_,oo Sf{Cn) ^ 

{K + l)e. Since K is fixed and e,j can be chosen arbitrarily small, we conclude that all rates R such 
that 

O^Ri< max ( min p-liminf -I(X''; YI^) - max p-limsup -I(X"; Z^) ) (13) 

are achievable, where V = {{X„}„^i : P[^Cn(X") ^ P] = l}. The achievability of the rates below the 
secrecy capacity in (12) is then obtained by introducing a prefix channel as in the proof of Theorem 2. 

We now turn to the converse part of the proof. Consider a sequence of wiretap codes {Cnjn^i achieving 
rate Ri for secrecy metric Sg. For n G N*, let V" denote the choice of a message uniformly at random 
in {1, 2"^!]]. By definition, for every n G N* and /c G [1, Kj, V" X" Y^Z^ forms a Markov chain 
and P[^c(X'*) ^ P] = 1. By the Verdu-Han Lemma [17, Theorem 4], we obtain 

R< min p-liminf -I (V";YIf). (14) 

By definition of the secrecy metric §6> we also have 

max p-limsup -l(V";Z^) = 0. (15) 

Combining (14) and (15), and maximizing over all processes {V"X"}, we obtain the desired result. ■ 
Although the secrecy capacity of a compound wiretap channel is identical to that of a mixed wiretap 
channel, note that the coding schemes achieving it may be fundamentally different. As mentioned earlier, 
a code designed for a mixed wiretap channel may not guarantee secrecy over the individual channels. 

Proposition 6. Given a memoryless compound wiretap channel with additive cost constraint P, all rates 
R such that 



0^R< max min I(V;Yjfc)- max I(V;Zfe) (16) 



are achievable for secrecy metrics Sj with i G p, 6], where 

77A|vx:VA;G[l,ifl V^X^ YfeZjk forms a Markov chain and E[c(X)] ^ P } 
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If the random variables maximizing (16) are such that, for all k G the moment generating 

functions of I(V; Y^) and c(X) converge unconditionally in a neighborhood of and are dijferentiable 
at 0, then the rates are also achievable for §i. 

Proof: The proof of Proposition 6 follows from steps similar to those used in the proof of Proposi- 



When applied to memoryless channels without cost constraint, Proposition 6 provides a generaliza- 
tion of [24, Theorem 1] for strong secrecy. Note that deriving secrecy from resolvability circumvents 

the enhancement argument used in [24], which is required to show achievability using capacity-based 
wiretap codes. Similarly, when applied to Gaussian compound wiretap channels with power constraint, 
Proposition 6 allows one to strengthen [25, Theorem 1]. 

Remark 10. The general result of Proposition 5 holds provided the number of channels K is fixed and 

independent of the number n of channel uses; nevertheless, in the special case of Proposition 6, for which 
we establish secrecy for metric Si, we can show that, for each A; G [1, K\ ^ {K + 1)2~^'''^ for some 
€k > 0. Therefore, Proposition 6 also holds if the number of compound channels grows exponentially 
with n as K = 2^" with (3 < minjtg|i^if] e^. 

C. Secret-Key Agreement from General Sources. 
As a last application, we exploit the result of Corollary 1 to analyze the fundamental limits of secret-key 



generation for a general source model. Specifically, we consider a discrete source lX,y,Z, {px" Y"Z" } 



with three components taking values in discrete alphabets. As illustrated in Figure 5, Alice and Bob 
attempt to distill a secret-key from their correlated observations X" and Y", respectively, by exchanging 
messages over a public authenticated channel with unlimited capacity. 

Definition 8. A (2"^, n) key-distillation strategy Sn consists of: 

. a key alphabet /C = [l, 2"-^]]; 

• an alphabet A used by Alice to communicate over the public channel; 

• an alphabet B used by Bob to communicate over the public channel; 

• a source of local randomness for Alice {TZx,Pv.x)> 

• a source of local randomness for Bob {Tly,PKY)y' 

• an integer r G N* that represents the number of rounds of communication; 

• r encoding functions fi : x x TZx — > A for i e [1, rj; 



tion 5 and Theorem 3 and is omitted. 
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• r encoding functions gi'.y^ x A^~^ x TZy — >■ B for i e [l,r]; 

• a key-distillation function Ka x x Tlx K.; 

• a key-distillation function : x x TZy — > /C; 
and operates as follows: 

• Alice observes n realizations of the source while Bob observes and Eve observes z"; 

• Alice generates a realization Vx of her source of local randomness while Bob generates Vyfrom his; 

• in round i G [1, r], Alice transmits ai = fi (x", b^~^, rx) while Bob transmits hi = gi (y", a*~^, r^); 

• after round r, Alice computes a key k = Ka (x", b^, Vx) while Bob computes a key k = (y", a^, Vy). 

The random variables corresponding to Vx and ry are denoted by Rx and Ry, respectively; for i e |1, r], 
those corresponding to messages Oj and bi are denoted by and Bj. The performance of a secret-key 



K / K 



S„ 



distillation strategy 5„ is measured in terms of the average probability of error Pe(iS„) = P 
the secrecy of the key Si(»S„) = Sj(pKZ''A'-B'-,J'KPZ'»A'-B'^) for i G [1, 6], and the uniformity of the key 
V{Sn) = 2'*-^-M(K). 

Definition 9. A key rate R is achievable for secrecy metric if there exists a sequence {<Sn}n^i of 
(2"^, n) key-distillation strategies such that 

lim Pe(5n) = 0, lim 8^(5^) = 0, lim U(5„) = 0. 

n->oo n— >-oo n— >-oo 

The secret-key capacity for metric Sj is defined as 

C^^ = sup{i? : R is an achievable key rate for metric Sj}. 

Theorem 4. The secret-key capacity of a discrete source (^X,y,Z, {px''Y"Z''}n>i) faf secrecy metrics 



Alice 
K 



Y" 



Bob; 
K i 




public authenticated channel 



Fig. 5. Secret-key agreement from general source. 
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max 




Sj with i e |2, 6] satisfies 

p-liminf -H(X"|Z") - p-limsup -H(X"|Y") 

n—^oo n—^oo 1^ 

p-Uminf -H(Y"|Z") - p-Umsup -H(Y"|X") 

n—^oo IT' n—^oo 

^ min (p-Uminf - 1 (X" ; Y"), p-liminf -I(X"; Y"|Z") J . (17) 

If the discrete source is i.i.d., the above inequalities hold for secrecy metric Si, as already known 
from [12], [13]. 

Corollary 3. The secret-key capacity of an i.i.d discrete source {X,y,Z,pxYz) far secrecy metric Si 
satisfies 

max (I(X; Y) - I(X; Z), I(X; Y) - I(Y; Z)) ^ C<'> ^ min (I(X; Y), I(X, Y|Z)) . 

Proof of Theorem 4 and Corollary 3: The achievability part of Theorem 4 is based on the construction 
of a conceptual wiretap channel as in [10]. Assume that AUce, Bob and Eve observe n realizations X^, 
Y" and of the source, respectively. Consider an arbitrary process {Ujjj^i such that Uj G X for 
j G [1, nj. Assume that Alice forms the signal U" © X" on the public chaimel, in which © denotes the 
symbol-wise modulo-A' addition. This operation creates a conceptual wiretap chaimel with input U'*, in 
which Bob observes the outputs Y" and U"©X" while Eve observes the outputs Z" and U"©X". From 
Corollary 1, the secrecy capacity of this conceptual chaimel for secrecy metrics Sj with i G [2,6]] is at 
least 

max (p-liminf ^ I(U"; Y'*, © X") - p-Umsup ^ I(U"; Z", U" © X") J . 

In particular, we can choose for {Ujjj^i an i.i.d. process such that, for all j G N*, Uj is independent 
of X"Y"Z" and uniformly distributed in X. Then, with probability one, 

,J'U"©X",Y"|U"(U"©X",Y«|U") 



I(U";Y",U"©X") = Iog 
= log 



PU"©X",Y'.(U"©X",Y^O 
Px-|Y-u-(X"|Y"U")py„|u„(Y"|U": 

Pu"ex"|Y"(U"©X"|Y")pY''(Y") 
1 



= log;.X"|Y''(X"|Y")-log|^l„, 

where the last inequaUty follows from pviu" (Y'^IU") = pyn (Y"), Px-IY-U" (X^IY^U'*) = px^lY" (X"|Y") 
since U" is independent of X"Y" and Pu»eX"|Y''(U" ® X"|Y") = |^ by the crypto lemma [6]. 
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Therefore, 

p-liminf -I(U"; Y", © X") = log \X\ - p-limsup -H(X"|Y"). (18) 
Similarly, one obtains 

p-Umsup -I(U"; Z", U'^ © X") = log \X\ - p-Uminf -H(X"|Z'*). (19) 

n— >-oo n— >-oo 

Combining (18) and (19), we conclude that any rate R such that 

R < p-liminf -H(X'*|Z") - p-limsup -H(X"|Y") 

is an achievable rate for the conceptual wiretap channel. Since this channel allows one to transmit 
uniformly distributed messages, R is also an achievable secret-key rate for the source model. The second 
lower bound is obtained by reversing the role of X" and Y" in the steps above. For i.i.d. discrete sources, 
a similar proof based on Corollary 2 in place of Corollary 1 shows that the result holds for metric Si as 
well. 

The proof of the converse is relegated to Appendix G. ■ 

Remark 11. Theorem 4 is easily adapted to a "channel model", in which X" is controlled by Alice and 
broadcasted to Bob and Eve through a channel (^V, 3^, ^, {pY"Z''|X"}n^i)- In this case, the bounds in 
Theorem 4 include a maximization over all possible distributions px^. 

Notice that the general form of the achievable key rates obtained in Theorem 4 involves conditional 
entropy; except in some special cases, such as i.i.d. sources, this is fundamentally different from the 
general form of achievable secrecy rates for wiretap channels obtained in Corollary 1, which involves 
mutual information. In particular, if p-liminf ^H(X'^) = p-limsup ^H(X"), then, 

n— >-oo n—^oo 

p-liminf -H(X"|Z'') - p-limsup -H(X"|Y") ^ p-liminf -I(X"; Y") - p-limsup -^X*"; Z*"). 

n->oo IT' n->oo n— >oo n->oo 

This distinction suggests that the fundamental mechanism for secret-key distillation, which one would 
have to exploit to design secret-key distillation strategies without relying on the existence of wiretap 
codes, is not Unked to resolvability; indeed, it has been argued that the mechanism behind secret-key 
distillation is the channel intrinsic randomness [26]. In that respect, despite its generality, the proof of 
Theorem 4 does not provide much insight into the design of practical secret-key distillation strategies. 
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VII. Conclusion 



We have analyzed several models of secure communication over noisy channels by exploiting the idea 
that the fundamental coding mechanism to ensure secrecy is related to resolvability. This approach has 
allowed us to establish results for generic channels and for stronger secrecy metrics than the usual average 
mutual information rate between messages and eavesdropper's observations. 

From a technical point of view, deriving secrecy from resolvability provides a conceptually simple 
approach to analyze the secure achievable rates of many models. Although we have limited examples of 
applications to mixed, compound, and wireless chaimels, the coimection between secrecy and resolvability 
is useful in many other settings. Examples of secure communication models for which deriving secrecy 
from resolvability simplifies the analysis include queuing channels [36], wireless channels with imperfect 
state information [35], runlength-limited channels [37], and two-way wiretap channels [38]. 

From a practical perspective, we believe that the connection between strong secrecy and resolvability 
is fundamental. We have provided evidence of this connection by proving that, for binary symmetric 
wiretap chaimels, sequences of random capacity-based wiretap codes, which are implicitly used in [1], 
[2], caimot achieve the strong secrecy capacity. Although this result has a limited scope, it is consistent 
with practical code constructions achieving strong secrecy rates [31], [39] and other approaches based 
on privacy amplification [12], [34]. 

Our results can be extended in several directions. For instance, the coding mechanisms for secrecy 
presented in Section IV for Shannon's cipher system and in Section V for wiretap channels can be 
combined without much difficulty using a coding scheme similar to that proposed in [40]. One could 
also further investigate the nature of the coding mechanisms for secrecy in secret-key agreement models. 
Some results along these lines are already available, for instance in [13], [26], [41]. 

Appendix A 
Technical Lemmas 

Lemma 7 (Chernov bound). Let X be a real valued random variable with moment generating function 
(?!)x : K — > M : s !->■ E[e*'^]. Let {'Xi}^=i be i.i.d. with distribution px- If 4^x converges unconditionally 
in a neighborhood of and is differentiate at then, 
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Lemma 8 (Basic properties of variational distance). Let X\, X2, and X3 be random variables defined on 
the same alphabet X. Then, 

^{px,;pxs) ^ V(pxi;px2) + V(px2;px3), 
and V(pxi;PX2) ^ V(pxiPX3;PX2X3) = [V(pxi,Px2|X3)] • 

Proof: The statements are immediate consequences of the definition of variational distance. ■ 

Lemma 9 (Data-processing inequality for variational distance). Let Xi and X2 be random variables 
defined on the same alphabet X. Let W^|x be transition probabilities from X to Z and define the 
random variables Zi and Z2 such that 

V(z,x) G Z X A' pziXi(z,x) = VFz|x(z|x)PXi(x) and pz^^^z^x) = Wz\x{z-\x)pxAt<-) ■ 

Then, V(pzi,pzJ ^ V(pxi,PX2)- 

Proof: Note that 



V(pZl,^>Z2) = Yl l^'Zi(z) -pz2(z)l = Yl 



X]pz|x(z|x)pxiW - 5^pz|x(z|x)^>X2W 



^ Y Zl^'zix(zlx) Ipxi(x) -PX2WI 



V(pxi,PX2 



Appendix B 
Proof of Proposition 1 

The fact that Si >z §2 and S4 >z S5 follows directly from Pinsker's inequality [27, Corollary p.l6]. 
Similarly, the fact that §2 h §3 and S5 y Se follows from [27, Corollary p. 18]; hence, we only need to 
prove that S3 ^ S4. 

Let e, 7 > 0. Assume that lim„_^oo S3(pmZ'»)PmPZ'») = 0, so that lim„_^ooP[I(M.; Z'*) > e] = 0. 
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Note that metric ^4:{pm.Z" , PmPZ") can be written as 

S4(PMZ",PmPZ") 

I(M;Z") 



= E 



= E 



n 

I(M;Z" 

n 



+ E 



n 

I(M; Z") 
n 



-1{I(M;Z'^) ^-e} 



+ E 



I(M;Z'' 



n 



l{-e< I(M;Z") ^e} 



1{€< I(M; Z*^) ^n(i? + 7)} 



+ E 



n 



1 {I(M; Z") >n(i? + 7)} 



Clearly, it holds that 



E 



I(M; Z"^ 



n 



1{I(M;Z") ^ -e} 



E 



I(M;Z'^ 



n 



l{-e< I(M;Z'^) ^ e} 



^ E 



|I(M;Z-) 



n 



<0, 



l{|I(M;Z")Ke} 



n 



and 



E 



I(M;Z^ 



n 



-l{e< I(M;Z") ^n(i? + 7)} 



^ (i2 + 7)P[I(M;Z") >e]. 



Following [7, p. 223], we can prove that 

■I(M;Z"] 



lim E 

n— >-oo 



n 



1{I(M;Z'^) >n(i? + 7)} 



Therefore, lim^^oo S4(pmz")J3mPZ") = and S3 >z S4. 



Appendix C 
Proof of Proposition 2 

Let Cn be the random variable that denotes a randomly generated code. The proof of the proposition 
relies on the following two lemmas. 



Lemma 10. There exists a\ > 0, such that, for n sufficiently large, 



liCn) ^ and S4(C„) ^ 2e, 



>l-2- 
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Proof: The existence of ai > such that P P*(Cn) ^ 2-5'"'' ^ 1- 2-^*1" follows from a standard 
random coding argument. Consider a code C„ such that P*(C„) ^ 2" 2'"". Then, for n large enough, 

S4(C„) = ^I(M;Z") = ^I(MX'*;Z'*) - ^l(X";Z''|M) 

= -I(X";Z") - -M(X"; |M) + -H(X"|MZ") 
n ^ ' n n 

^^Ce-{Ce-en) + R'KiCn) 

where (a) follows from Fano's inequaUty. ■ 
Let be the random variable with uniform distribution on Z^, i.e. for every G Z'^, pzn {z'") = 

Lemma 11. There exists j3, 0:2 > 0, such that, for n large enough 



V(pzn,PZ") ^2- 



> 1 - 2 



Proof: This result follows, for instance, from [42, Lemma 19]. ■ 
For n G N*, let C„ denote a randomly generated code such that 

p:(C„) ^ 2-5'"^ S4(Cn) ^ 2e„, andV(p2",PZ") ^ 2-^^". (20) 

For n large enough. Lemma 10 and Lemma 1 1 guarantee that this occurs with probability at least 1—2-"^" 
with 0:3 = imin(ai,a2). With a shght abuse of notation, we also let C„ C X'^ denote the codebook 
and let /-^ : C„ — > All be the restriction to TWi of the inverse mapping of /„. Define new functions 

<j)n and tl)n as 

: C„ Ml and : x >ti ^ C„ 

x"" ^ fn^ix"") (2;",m) ^ fn{m,hn{z'',m)) 

These functions define a source code for the compression of the source X" G C„ (the choice of codewords 
uniformly at random in the code) with Z" as correlated side information at the receiver, whose probabihty 
of decoding error is P*(Cn). We now leverage the results obtained by Hayashi [43] and generalized by 
Watanabe et al. [41] that establish a tradeoff between probability and error and resolvability for source 
coding of arbitrary sources. Combining [41, Theorem 6] and the proof of [41, Theorem 7], we obtain, 
for any 6 > , 

Vn G N* Vl{Cn) + S2(C„) ^ 1 - (2-^^+1 + Pxnz„ [^0]) , (21) 
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with 



Note that \M\ \ = 2"(i-H''(*^)-«") and Px"(^") = \mA\M',V Therefore, 



n—byjn 96 



X"Z" 



X"Z" 



l-^ll 

\Mi\ 



\Mi 



log- 



Pz"|X"(Z"|X-) 



PZn(^") 



^6v^ + n(l-M,, ((52) -en) 



X"Z" 



log 



PZ.|xn(Z"|X-) 
Pz»(Z") 



^ -b^ + n(l - Hfe {62) - 



= Px"Z"[2n]+Px"Z"[2n], 

where we have defined 

Qt = (^", ^") eCnXZ^: log „ ^ ±bVn + n(l - H„ (^2) - e„) 



Pz-C^'*) 

We analyze Px''Z"[2n] ^X'^z^lQn] by introducing the sets 



At = <{x'\z'')£ CnXZ^: log - 



^ ±26V^ + n(l-H6 (52)-en) 



(x", z") G C„ X : log ^^^45 < 



and P„=Ux",^")G 



C„xZ":log^^4^!>-^V^|- 



Using the law of total probability and the fact that Q+ r\BnC A^, we now upper bound Px^ifn [Qn] 
as follows 



^XnZ. [Q+] = PxnZn [O-t ^ ^n] + Px^Z" [Q^ ^ ^\ 

^Px.Z"[-4n]+Px"Z"K] 



(22) 
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For n large enough, 



(a) 1 ^ ^ \zr 

2^(log|Z|+/3)2-/^" 





(23) 



where (a) follows from [13, Lemma 1] and (5) follows from the fact that x i-> a; log is monotonously 
increasing for x small enough. In addition, note that 



1 Pz"ix42:"|X") 

log /^J^ ^ 26 + - Hfe (52) - 



Since the noise is additive, we have Zj = Xj + Ej for ? G [1, n] where {Ej}i^i is i.i.d. with distribution 
Pe ~ 'B((52) and independent of {Xi}i^i; hence, Pzn|xn (Z"|X") = nr=i^'E(Ei) and 



, Pz.|x^(Z-|X") " p,(E,) 
log — — Tw:^^ — = Z^log ^ 
•t=i 



L/2 



The random variables in the sum are i.i.d. with mean 1 — H5 ((52), variance cj > 0, and third moment 
p < 00. From the Berry-Esseen Theorem [44], there exists a universal constant c > such that 

1 



/2^ 



e 2 dx + 



c /? 



(24) 



Similarly, using the law of total probability, the fact that n P„ C Qn and the inclusion- 

exclusion principle, we lower bound Px^z^i^n] follows 



C"Z"[2. 



Xnz- 



^Px''Z"M+IPx'^Z'>[2^n]-l 



(25) 
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Note that, 



^ 2-^v^. (26) 
and, following the reasoning leading to (24), 



2b v^€ 

c p 



x.z^ [^-] ^ ^ J_ e—dx --j=-^^. (27) 



Combining equations (22)-(27), we obtain 



2b y/ne-n . 

r~ ' «"^dx + ^4 + ^ (log \Z\ + /3) 2-/^" + 2-^^ 



^ + + ^ (log |Z| + /3) 2-^" + 2-^^. (28) 

Combining (28) with (21), and using the assumption lim„_^oo Pe(^n) = from (20), we have 

V6>0 lim S2(Cn) ^ 1 

n^oo crV27r 



Therefore, there exists r) > such that, for n large enough, S2(Cn) ^ ??• Notice that Proposition 1 
immediately implies that there exists r]* > such that lim„_^ooSi(Cn) ^ r)*- 

Appendix D 

Lemmas used in the Achievability Proof of Theorem 2 

The following notation is used throughout this appendix. We recall that U", X", Y", Z" are the random 
variables defined by the random code generation with distribution given in (7). For any {k, I, m) G 
[[l,Mo]] X [[l,Mi]] X [[1,M(]], the random variables representing the codewords and x^';^ obtained 
with the random code generation are denoted by U^* and X^^^^^. 

The random variables that correspond to the use of a specific code C„ are denoted by U", X", Y", Z" 
with distribution given by (4). The channel outputs that correspond to the transmission of and x^^^ 
are denoted by Y^;^, and respectively. 
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A. Proof of Lemma 1 
By symmetry of the random code construction, we have 

^ Mo Ml M[ 

E[Pe(C„)] = J^JfJ^ E E E ^ [Pe(Cn|Mo = k,Mi = I, M[ = m)] 
° ^ 1 A;=l 1=1 m=l 

= E [Pe(Cn|Mo = 1, Ml = 1, M[ = 1)] , 

which we can analyze in terms of the events 

Eiik) ^ {(U^, Yrn) € TTIMo = 1, Mi = 1, Mi = l} 
E2ik) ^ {{Ul Z^n) e 7?|Mo = 1, Mi = 1, Mi = 1} 
E,ik,l,m) ^ {iUl,Xli^,Y^u) G 7i"|Mo = l,Mi = l,Mi = l} . 

The average probability of error can then be written as 

E[Pe{Cn)]=E F Et{l)u\jEi{k)UE2{k)u\jE2{k)UEl{l,l,l)U \J E^{l,l,m) 

where the expectation is with respect to {U^} and {X^^^} for {k, I, m) G [1, MqI x [1, x [1, M{1. It 
follows from standard arguments (see for instance [7, Chapter 3]) that E[Pe(Cn)] < e for n large enough 
provided 

I log Mo ^ p-liminf il(U"; Y^) - 27 

n— >-oo 

I log Mo ^ p-liminf il(U"; Z") - 27 (29) 

ft— 5- 00 

^ logMiM( ^ p-liminf ^I(X"; Y"|U") - 27. 

n— >-oo 

B. Proof of Lemma 2 

We start by developing an upper bound for S2(Cn) that will be simpler to analyze. First, we have 



E 



U"Mi 



V 



(pZ''|U"Mi)PZ"|U' 



Next, we use Lemma 8 to further bound §2(^71) as follows. 

)+v(pz»|un,Pzn|un)] 



< E 



U"Mi 



(PZ"|U"MHPZ''|U'') V(^PMii?Z''|U",PZ''Mi|U") 



= 2Eu„^4^ v(^P2"|U''Mi>PZ"|U") • 



(30) 
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Notice that the term in brackets on the right hand side is a variational distance between the following 
two distributions: 

^ v^m; 1 



• PZ"|U"=u.J,Mi=«l 



klm 



I, which represents the distribution induced at 



the eavesdropper's channel output by the M[ codewords {''^fcaiieli.Mii selected with a uniform 
distribution. 

• PZ"\U"=ui{^"') = Z^x" ^Z''|X"(z"|''c")j>x''|U''=u.^ which represents the distribution induced at 
the eavesdropper's channel output by an input process with distribution Px''|U''=ug(^")- 
Therefore, a sufficient condition for §2(^,1) to vanish is that, for every pair {k,l) G [l,Mol x [l,Mi], 
the variational distance between the two distributions vanishes as well. This is possible if each set 
of codewords {Xfc/Jie[i,M;| approximates the same process with distribution Pz"|U"=u^! (2^") at the 
eavesdropper's output, which is exactly what the concept of channel resolvability reviewed in Section 11 is 
about. In other words, a sufficient condition to guarantee secrecy is for each sub-codebook {'^^ii\ieli,Mi\ 
to be "resolvability code" . 

We estabhsh the existence of such codebooks with a random coding argument following that used 
in [5]. The presence of a common message makes the proof slightly more involved but the steps remain 
essentially the same. On taking the average over C„ for both sides of (30), we obtain 



(31) 



By symmetry of the random code construction, the inner expectation in (31) is the same for all values 
of U" = and Mi = Z; hence, we have 



Ec„[S2(C„)] ^ 2Ec„[v(p2n|u"=urM,=i,Pz'.|U"=ur) 



Let T > 0. On using [7, Lemma 6.3.1] we finally upper bound (32) as 

Ec„[S2(C„)] ^4r + 4A„ 



(32) 



(33) 



with Ar. 



Mr 



Z"|U''=UJMi=l 



PZ''|U''=UrMi=l(^ ) ^ 
log T=^. > T 
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Note that the expectation over reduces to the expectation over and {X.iij}je[i,Mi}- Writing A^^ 
exphcitly, we obtain 

An= J2 PU-i'^l) Yl PX"|U"WllK)--- J2 PX-IU-WlM^?) 



E/ n\ HI fi ^'Z"|U"=u.?Mi=l(^"') 1 



-'■ m=i ix"eW" xiiieA"" xiiM^eA"" 

ETiA / ni n Nil fi ^'Z''|U"=u;'Mi=l(2^") 1 



(6) 



E ^'u-K) ^ ^3x"|U"(xn2m')--- E Px-\u4^iiMi\0 



> r 



E ^^U"«) J]] Px-|U"Wi2|u?)--- Yl PX-|U-WiMilO 

/ n n 1 Ji / ^ PZ"|X"U" (^^"IXiimU") \ 

XiiieA-" z"eZ" y m=l -f^ |u V 11/ J 

where equality (a) follows from the definition of Pz^\U"=xi^Mi=i,c„{^^)' equality (6) follows by re- 
marking that all codewords are generated according to the same density Px^lU" equality (c) follows 
by noting that 

• W^Z"|X"(z"|x;\i)px"|U"(xni|i^i) =Pz"X"|U"(2^">^iii|t^i) according to (7); 

• for any u" such that Px'*|U"(x-iiml'"-i) > 0, 

Pz"|U"=urMi=i(z") = ir77 E ^z-ix-l^^lxn™) = V77 E ^^z-lx-u-l^^lxn^O. 



m=l ^ m=l 



Setting p = ^^y^ we obtain 

M: 



I E 



1 f^PZ"|X"U"(z1x^,^U-) 



> r <;4> 



J_ A Pz.|xnu>.(z"|x?,,uy) V J_ A pz»|x.un(z"|x?,^uy: 
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Therefore, 

An ^ Pu-X-Z" 



J exp log 



Pz„|x.un(Z"|X"U") 
Pz.|un(Z-|U") 



+ Pu''Xri2...XiiM,'Z'' 



— > exp log > 1 + P 

^ pz"iu"(z"K) y 



-I(X";Z"|U") > -logM( + -logp 
n n n 



Xfi2----^11M' lU^Z" 



1 / 

M[5:«-p (log 



> 1 + P 



(35) 



By (34), note that, conditioned on U" = u", {X.iij}jel2,Mi} i-i-d with distribution Px''|U''=u" 
independent of Z". To analyze the second term on the right-hand side, let us define 



Let us introduce the random variables 



1 ^ A Pz'.|X"U"(z"|X;\^,u«)\ 
— 7 > exp log — -^^T ] > 1 + p 



D^"(z",u'^)^exp(loi 



.PZ"|X"U"(Z"|X^1^,U" 



forjG [2,M0 



E^"(z'^, u'^) ^ Dj"(z", u")l {D^"(z", u") ^ M{ } for j e [2, M(l 



Mi 



J=2 



i=2 

For a fixed z", u", the random variables {D"(z",u")}jg|2,M;] i.i.d. by construction, and so are the 
random variables {E"(z",u")}jg|2,M{]- By the law of total probabihty, 

Bn{z^,u^) = Pxr,,...x^,^,|U"=u" [^Mdz'\u'^) >l + p] 
^Px^,....x;',„,|m=^.4GM^(z^u-) >l + p] 



+ Pxj,,...X" ,|U"=u4Gm|(z",u«) / FMi(z",u")] . 



(36) 
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We first bound Dniz^jUP) as follows. Note that 



D„(z",u")^Px.,...x" ,|U 



m: 



U{E-(z",u")7^D^"(z-,u-)} 



j=2 



^ M{Pxr,,|U"=u" [D^(z",u«)>M(]. 
Therefore, on taking the expectation over Z'^U", we have 

Ez"u4^n(Z",U")] 

= «;EEEj'--(-^-")l'xiu.(x"|u")l{exp(log?^^;^^ 

< E E E j'z""- (^". ""jpx-iu- (x-'iu-) 



Z" U" X" 



PZ"|X''U"(Z"|X",U")\ r / ;PZ"|X"U"(Z"|X",U") 
exp log , ^, — 1 <^ exp log 



-EEE^z^^x....^(Ax^u■,,{e.p(.o /7;;'g^;;"' )>M;} 



1 PZn|Xnu4Z"|X-U") ^ 1^ ■ 



Finally, we bound Ez"U"[C'n(Z"^,U")] using Chebyshev's inequality. Note that 

Ex^.,...x;«^„,|U"=u4Gm((z",u")] 



J=2 



Exr,,|un=u"[t?(z",u-)] 



2^PX"|U"(^ |u")exp I log ' > — ' 



= X;pxn|znu"(x"|z",u'^)l |exp {log 



PZn|Un(z"|u") 

PZ-IX-'U^l^; |X ,U 



1 < exp I log 



PZ„|Un(z"|u") 



PZ„|Un(z'»|u") 



^ 1, 
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where the last equality follows from Bayes' rule. Define 



Var(GM((z",u")) ^Ex.^,|un=u4(GMi(z",u») -Exn^|Un=u4GMK2",u")])' 



Note that 



Var(GM((z",u")) = ^Var(E^(z",u")); 



hence, on applying Chebyshev's inequaHty, we have 



^ iPxr,,...x^^„,|U"=u" 

^ ^Var(GM^(z^u-)) 



Gmi {z\ u") - Ex. ...x;^, |U"=u" [Gm( {z\u-)\ > p 



Therefore, on taking the expectation over Z^M^, we obtain 

Ez.U"[C„(Z^U")] = 5;5^Pz.u.(z«,u«)Pxw....x5',„,|U"=u4GMi(z",u") >l + p] 

^ EE^'^''U"(z^u")-^Var(E»(z^u")) 



(38) 



Finally, note that 
1 



fPz ^lx^ur^ (z"|x", u") y r Pz.^|x.^u. (z"|x", u») ^ ^, 

PZ"|U"(Z"|U-) J 1 PZ"|U"(Z"|U") ^ 1 



1 ^71 2" ^ 

^ V W„ r." ^. /z-|x-u4z-|x»,u») [ Pz-|x-u-(z"|x",u-) /I 



Eu''X"Z 



U" X" z 

1 



^ exp I(X"; Z"|U'^)1 {exp I(X"; Z"|U") ^ M[} 



X^Z" 



-I(X";Z"|U") ^ 

n n 



(39) 
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Combining (35), (36) (37), (38) and (39), we obtain that for any r > 

Ec„[S2(C„)] ^4r + 4Punxnz 



n n n 



+ 4Pu.X" 



n n 



+ 



4 • 2-"T 



+ 



Therefore, Ec„[S2(C„)] < e for n large enough provided 



U"X"Z'' 



il(X";Z"|U")^i^^-7" 
n n 



^ logM[ ^ p-limsup -I(X"; Z"|U") + 2j. 

IT' n—^oc 



(40) 



(41) 



Appendix E 

Lemmas used in the Converse Proof of Theorem 2 

A. Proof of Lemma 3 

Following the proof of the Verdu-Han Lemma, one can easily show that 



3Pe(Cn) ^ 



from which the lemma follows. 



T(U";Y-) ^i?o-7 
or ^I(U";Z") ^i?o-7 
or il(W";Y"|U") ^ iili -7 



3-2 



-n-y 



(42) 



B. Proof of Lemma 4 

To prove Lemma 4, note that, with probabihty one, 

il(Vr;Z") = il(Vr;Z"U'^) - il(W";U"|Z") 

= -l(Vr;Z"|U") - -I(W";U"|Z") 

n ' n ^ ' 

= -l(Vr;Z"|U") - -H(U"|Z") + -H(U"|VrZ") , 
where the second inequality follow from the independence of and U". Consequently, 
lim S6(C„) =p-hmsup -l(W";Z'') 

^ p-limsup -I(W";Z"|U") - p-limsup -H(U"|Z'') +p-liminf -H(U"|W"Z'') . 

n— >-oo n— >-oo ^ n-^oo ^ 
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Note that p-Uminf ^H(U"|W"Z") ^ and that (42) implies 

n— >-oo 



3Pe(C„) ^ P 



n 



3 • 2-"T. 



Since lim^^oo Pe(Cn) = and 7 > can be chosen arbitrarily small, we have p-limsup ^H(U"|Z") = 0. 

n— >-oo 

As lim^^oo Sel^n) = 0, we finally obtain 

p-limsup -l(Vr;Z"|U'*) = 0. 

Appendix F 
Proof of Theorem 3 

We prove Theorem 3 with minor modifications of the proof of Theorem 2. Specifically, we establish 
secrecy for Si by showing that there exist sequences of codes {Cn}n^i for which S2(Cn) decreases 
exponentially fast with n and by using [13, Lemma 1] to obtain an upper bound for Si(C„). We handle 
the power constraint by using an appropriate distribution during the random code generation process as 
in [7, Section 3.2]. We note that a similar proof has been used by He and Yener in [45]. 

Let 7, 5, e > 0. Let W be an arbitrary discrete alphabet and fix a distribution p^. on U. Fix a conditional 



distribution pj^|Q on x W such that E 
joint distribution 



c(X) ^ P - (5. Let U", X", Z" be the random variables with 



1=1 



We assume that U'^,X'^,Z" are such that the moment generating functions of c(X) and I^X;Z|uj 
converge unconditionally in a neighborhood of and are differentiable at 0. 
Define the set Vn as 



Vn 



x" G Af" 



1 " 



X,:) ^ P 



i=l 



Lemma 7 shows that there exists > such that P j^X** G 
7n = 1 - 2~"^. Define the set Gn C as follows: 



^1 — 2 In the sequel, we define 
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Upon using Markov's inequality, we obtain 





^ Gn 


















IPx"|U" 






XT 



as 

2"^ 



^ 2—n{as—'^) 
as 



2"^ 



Now, we define the random variables U",X",Z" as follows. First, 

else. 



By construction, we have 



7n 



(43) 



Next, 

By construction, we have 



else. 



7n 



(44) 



Finally, 

V(z",x-,u") eZ^xX^xGn pznxnu™(z",x",u") = W^z''|x"(z"|x")px''|m(x"|u")pun(u'^) . 

(45) 

We repeat the random coding argument in the proof of Theorem 2 with the distribution px^U" defined 
by (45). 



Lemma 12 (Reliability conditions). 

' Ro ^ min (^I (^U; y) - 27, 1 (U; Z 
^ Ri + R[^l(x-Y\U^ -27, 



27 



lim E[Pe(C„)] ^ e. 
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Proof: Following [7, Proof of Theorem 3.6.2], one can show that 

p-liminf -I(U"; V*) ^ l(U; y) , p-liminf -I(U"; Z") ^ l(U; z) 
and p-hminf -I(X"; Y"|U") ^ l(x; Y|u). 
Hence, the result follows directly from Lemma 1. 

Lemma 13 (Secrecy from resolvability conditions). There exists ag^^ > 0, such that 

R[ ^ lfX;Z|u) +27,^ lim E[Pe(C„)] ^ 2' 

Proof: Note that (30) still holds. Upon using Lemma 8, we obtain 

Ec„[S2(C„)] ^ 2EcJv(pz.|u.=ufM,=i,c„>;^Z"|U"=u?' 



-CCS,- 



< 2Er 



' (f^Z" I U" =Ur Ml = 1 , C„ ' ^'Z" I U" =Ur ) 



+ 2Ec 



(46) 



First, we bound the second term on the right-hand side of (46). For all u" G Qn, 

v(p2''|u''=ur^^z"|u"=uf) 

(«) / \ 
^ V(^Px"|U"=ur^'x"|U"=urj 

= 2 sup |Px"|U"=ur[^]-Pxn|Un=un[.4] 



ACX" 



^^^P„ E (|Px"|U'.=u?[^nK] -Px"|ti„=^„[i3np;;] -Px.|ti„=,„[^nK]|) 



tSe{A,A''} ^ \ln/ J 

= (;l-l) + (l-7n), 
In 

where (a) follows from Lemma 9 and (6) follows from the definition of Px"|U'' in (45) and the bound 
in (44); therefore, for n large enough there exists fii, > 0, such that 



Er 



^ 2- 



(47) 
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We now bound the first term on the right-hand side of (46). Applying [7, Lemma 6.3.1], we obtain 



2Er 



'(pZ»|U"=UrMi=liPz5'|U''=Ur) 

^ 4t + 4P2„|un=urM,=i 



PZ"|U"=UfMi=l(2") 
log 7^-^ > T 



(48) 



Note that (48) is similar to (33), and the only difference is the presence of |Q instead of pz\u in the 
denominator; using the definition of pz"X"U" in (45), the bounds in (43) and (44), and repeating the 
steps leading from (33) to (40), one obtains after some calculations 



2Ef 



V 



(pZ"|U"=U5'Mi=l!Pzr|U"=U5' 



+ 



7„ 



2^U"X"Z'' 



n 



^ 4r + 



I(X'';Z"|U" 



,,2 U"X"Z'' 



n 



I(X";Z"|U") > 



log M( log p 



n 



+ 



n 



logM' 



n 



4 . 2-"T 



+ 



(7„p + 1 - 7n)2 

\ / n 



If ^ logM( ^ I^X; Z|uj +27, then Lemma 7 guarantees there exists > such that 



7 



(49) 



Unx-Z" 



n 



I(X'*;Z"|U" 



n 



7 



(50) 



Set r = 2"''" for some r/ such that < 2?7 < min(7, a^)\ note that p = ^^^'"^"^ + 0(2-^""). Therefore, 
for n large enough. 



1 , 1 

-logp^-7, — 

n -inP + 1 - 7n 



^ 2 • 2"", 4^ ^ 2. 
74 



(51) 



Consequently, combining (46), (47), (49), (50) and (51), we obtain for n large enough, 

Ec„ [S2(Cn)] ^ 4 • 2"''" + 8 • 2~"^" + 8 • 2"''-^" + 16 • 2-(T-2r;)n _^ . 2-(a7-2r?)n _^ 2 • 2"'^''". 

Therefore, for n large enough, there exists a-y,5 > such that E[S2(C„)] ^ 2"°^^ ■ 
Using Markov's inequality and for n sufficiently large, we conclude that if 

i?0 ^ min (U; y) - 27, 1 (^U; z) - 27) 
i?i ^ l(X;Y|u) -l(^X;Z|u) -47, 

then there exists a specific code C„ such that Pe(Cn) ^ 2e and S2(C„) ^ 2~^^". Using [13, Lemma 1] 
with n large enough, we obtain Si(C„) ^ 2"^^^'^" for some ^-^^ > 0. 
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Appendix G 
Converse Part of Theorem 4 

In the following, all equalities should be understood to hold with probability one. First, note that 

I(Y";X"|Z") I(Y"Ry;X"Rx|Z") (52) 

= I(Y"Ry;X"RxAi|Z") - I(Y"Ry; Ai|X"RxZ") 

I(Y"Ry; AilZ'^) + I(Y"Ry;X"Rx|AiZ") 

= I(Y"Ry;Ai|Z") + I(Y"RyBi;X"Rx|AiZ") - I(Bi; X"Rx|AiY"RyZ") 

I(Y"Ry; AilZ'*) + I(Bi;X"Rx|AiZ") + I(X'^Rx; Y^RyIZ'^AiBi), (53) 

where (a) follows from the independence between Rx, Ry and the source, (b) follows from I(Y"Ry; Ai |X"RxZ'*) 
and (c) follows from I(Bi; X''Rx|AiY"RyZ") = 0. By induction, we obtain 

r r 

I(X";Y"|Z") = I(X"Rx;Y"Ry|Z"A^B^) + J]l(Y"RY;Ai|Z"A^-^B^-i) + J]l(X"Rx;Bi|Z"B^-W). 

i=l i=l 

Next, notice that 

I(X"Rx;Y"Ry|Z"A^B^) = I(X"RxK; Y"Ry|Z"A''B'^) - I(K; Y"Ry|Z"X'*RxA''B'-) 

I(K;Y"Ry|Z"A^B'') + I(X"Rx; Y"Ry|KZ"A'^B'') 
= i(k; Y"RyK|Z"A''B'-) - i(k; K|Z"Y'*RyA''B'-) + I(X"Rx; Y"Ry|KZ"A''B'-) 

^= i(k;K|Z"A''B'-) + i(k; Y"Ry|Z"A''B'^k) + I(X"Rx; Y"Ry|KZ'*A''B'-), 

(54) 

where (a) follows from I(K; Y"Ry|Z"X"RxA''B'') = and (6) follows from i(K; K|Z"Y"RyA''B'') = 0. 
Finally, 

i(k;K|Z"A''B'') = i(k;KZ"A''B'') - I(K; A^'B^Z'^) 

= H(K) - h(^K|KA''B''Z") - I(K; A^B^Z"). (55) 
Combining (53), (54) and (55), we obtain 

H(K) = I(X"; Y"|Z") + h(k|KA''B''Z") + I(K; A'^B'^Z") - i(k; Y"Ry|Z"A''B''k) 

k k 

- I(X"Rx; Y"Ry|KZ"A''B'-) - J]] I(Y"Ry; Ai|Z"A^-^B^-i) - ^ l(X"Rx; B^IZ^B^-^A^) 

i=l 1=1 
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and, consequently, 

p-liminf -H(K) ^ p-liminf -I(X"; Y"|Z") +p-limsup -h(k|KA''B'^Z") +p-limsup -I(K; A^B^'Z") 
- p-liminf I (k; Y"RY|Z"A''B^k) - p-liminf I(X"Rx; Y"Ry|KZ"A^B'') 

T V 

- J^p-liminf -I(Y"; Ai|Z"A*-^B^-i) - p-liminf -l(X"; BjlZ'^B^-^A^) 

. 1 n—^oo Tl . n—^oo Tl 

1=1 1=1 

By assumption, lim„_^oo ^eiSn) = 0, hence p-limsup ^I(K; A''B''Z") = 0. Similarly, since lim„^ooPe(«5n) = 

n— >oo 

0, the Verdu-Han Lemma ensures p-limsup K|KA''B''Z" j = and, since lim^^oo = 0, 

p-liminf ^H(K) = R^. In addition, note that 

n—^oc 

€ [1, rj p-liminf ^ I(Y"Ry; Ai|Z"A^-^B^-i) ^ and p-hminf ^ l(X"Rx; BjlZ^^B^-^A^) ^ 

n—^oo Tl n—^oo Tl 

p-liminf I ('K;Y'*Ry|Z"A'^B^k) ^ and p-liminf I(X"Rx; Y"Ry|KZ"A'^B^). 

Therefore, 

Rk ^ p-liminf -I(X"; Y"|Z"). 

n—^oo Tl 

Repeating the same argument starting from I(X"; Y") in place of I(X";Y"|Z"), we obtain 

illfc ^p-liminf -I(X";Y"). 

n— >oo Tl 
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